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ABSTRACT 

The  Systems  Simulation  Centre  of  the  DSTO  is  currently  considering  the  implementation 
of  a  commercial-off-the-shelf  software  package  called  SensorVision  to  fulfil  the  scene 
generation  function  of  an  infrared  hardware-m-the-loop  (HIL)  system.  Before  the 
software  can  be  used  for  the  intended  application,  there  is  a  need  to  verify  and  validate 
the  SensorVision  models  to  ensure  that  the  generated  scenes  are  sufficiently  realistic  for 
HIL  simulation  purposes.  This  report  discloses  the  results  and  conclusions  of  a 
validation  effort  focused  on  the  SensorVision  thermal  emission  model,  which  includes 
both  the  siurface  temperature  prediction  of  objects  and  the  thermal  radiance  calculations. 
It  is  shown  that  the  thermal  emission  model  employed  by  SensorVision  has  errors  that 
can  affect  the  level  of  realism  associated  with  the  generated  infrared  images.  Unrealistic 
scenes  can  cause  spurious  HIL  simulation  results,  since  these  infrared  images  are  used 
as  the  primary  stimuli  for  the  system  being  tested. 

A  procediure  for  providing  confidence  in  HIL  simulation  results  is  recommended, 
involving  general  guidelines  for  simulation  construction  and  post-processmg  operations 
to  provide  users  with  image  error  indications. 
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Validation  of  the  SensorVision 
Thermal  Emission  Model 


Executive  Summary 

The  validity  of  models  used  in  the  SSC's  (Systems  Simulation  Centre)  HIL 
(hardware-in-the-loop)  facility  at  the  DSTO  has  become  an  issue  of  recent  concern.  The 
results  of  a  HIL  simulation  has  only  Limited  usefulness  if  there  is  low  confidence  in  the 
ability  of  the  HIL  scene  generators  to  produce  realistic  virtual  environments.  Both  the 
target  signature  and  background  clutter,  as  well  as  any  other  significant  signature  effects, 
need  to  be  faithfully  reproduced  by  computational  models.  Confidence  in  HIL 
simulation  results  therefore,  in  part,  requires  confidence  in  the  scene  generator  models. 

In  this  report,  the  validation  effort  conducted  on  a  COTS  (commercial-off-the-shelf) 
software  package  called  SensorVision  is  described.  The  SSC  intends  to  employ 
SensorVision  for  the  purpose  of  real-time  image  generation  for  HIL  simulations 
involving  imaging  IR  seekers.  The  physics-based  model  used  by  SensorVision  to 
calculate  the  emitted  radiance  from  solid  objects  due  to  thermal  emission  processes  is 
scrutinised,  and  the  temperature  prediction  model  is  validated  against  experimental 
data.  The  two  main  outcomes  of  this  work  are:  a  figure-of-merit  parameter  was 
developed  to  indicate  the  accuracy  of  the  thermal  emission  model  for  any  given 
simulation;  and  it  was  shown  that  the  current  version  of  the  temperature  prediction 
model  is  not  appropriate  for  Australian  climates  and  conditions. 
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1.  Introduction 

1.1  Infrared  Scene  Generation  and  SensorVision 

The  SSC  (Systems  Simulation  Centre)  is  currently  in  the  process  of  upgrading  the  HIL 
(hardware-in-the-loop)  facility  at  the  DSTO  (Defence  Science  and  Technology 
Organisation)  to  enable  the  test  and  evaluation  of  imaging  IR  (infrared)  seekers.  The 
upgrade  cor^sists  of  two  main  components: 

•  the  acquisition  and  integration  of  an  IRSP  (infrared  scene  projector)  into  the 
existing  system;  and 

•  the  development  of  an  IRSG  (infrared  scene  generator)  capable  of  real-time  IR 
image  generation. 

The  proposed  HIL  system  is  shown  schematically  in  Figure  1-1.  The  subsystem  of  main 
interest  in  this  report  is  the  IRSG.  The  functions  of  the  IRSG  are  two-fold:  first,  it  is 
responsible  for  creating  and  maintaining  the  3-D  (three-dimensional)  virtual 


Controller 

Figure  1-1:  IR  HIL  system. 
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environment  within  which  the  seeker  is  to  be  immersed;  and  second,  it  renders  that 
environment  into  the  2-D  (two-dimensional)  images  used  to  stimulate  the  seeker  via  the 
IRSP. 

The  requirements  of  the  combined  IRSG  and  IRSP  modules  are  demanding  in  that  the  IR 
scenes  used  to  stimulate  the  seeker  must  be  realistic,  and  the  image  framerates  can  be 
extremely  high  (in  excess  of  100  Hz  for  some  air-to-air  seekers).  The  former  criterion  is 
somewhat  ambiguous  because  it  is  dependent  on  the  SUT  (system  imder  test).  The 
scenes  only  need  to  be  realistic  enough  such  that  the  SUT  performs  in  a  marmer 
consistent  with  a  real-life  engagement.  Given  the  diversity  of  current  and  future  IR 
seekers  that  may  be  tested  within  the  HIL  facility,  the  policy  adopted  by  the  DSTO  is  to 
ensure  that  the  scenes  used  to  stimulate  the  seeker  are  as  realistic  as  possible  given  other 
critical  constraints  (such  as  cost,  effort,  and  the  requirement  for  real-time  performance). 

The  quality  of  scenes  and  the  rate  of  image  generation  is  initially  controlled  by  the  IRSG. 
The  IRSP  may  then  degrade  the  scenes  and  affect  the  framerate,  depending  on  the 
capabilities  of  the  unit.  In  addition,  various  IRSP  induced  artifacts  may  occur.  This  report 
will  only  consider  the  IRSG. 

The  ERSG  system  that  is  currently  being  evaluated  in  the  SSC  is  a  COTS 
(commercial-off-the-shelf)  based  system.  The  decision  to  pursue  a  COTS  solution  was 
primarily  motivated  by  cost  considerations.  The  ptuchase  and  maintenance  of 
commercial  products  is  generally  more  cost  effective  than  the  in-house  development  of 
custom  products.  The  IRSG  system,  which  is  shown  in  Figure  1-2,  uses  an  SGF*^  Onyx2^'^ 
computer  platform  with  an  InfiniteReality2T^i  graphics  engine.  A  real-time  visual 
simulation  tool  called  Vega’^^  is  used  to  control  the  simulations  and  perform  the 
rendering  operations.  Vega  is  built  upon  the  graphics  languages  OpenGL™  and 
Performer'^'^.  SensorVision^'^  is  a  Vega  add-on  module,  which  enables  Vega  to  perform 
IR  simulations.  SensorVision  contains  the  physics-based  models  required  to  compute  the 
radiometric  ER  images  associated  with  the  virtual  environment.  Both  Vega  and 
SensorVision  are  distributed  by  MultiGen-Paradigm™  Incorporated. 

SensorVision  is  a  real-time  IR  signatiue  prediction  package,  which  uses  various 
approximations  to  enable  real-time  computations.  The  nature  of  these  approximations 
are  described  in  detail  in  reference  [1].  This  report  focuses  on  the  validation  of  the 
SensorVision  thermal  emission  model,  which  includes  both  the  surface  temperature 
prediction  of  objects  in  the  scene  and  the  thermal  emission  radiance  computation. 
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SensorVision  Simulation 


Figure  1-2:  COTS  based  IRSG  system. 


1.2  Verification  and  Validation 

The  concept  of  verification  and  validation  (V&V)  is  frequently  encountered  in  a  range  of 
different  engineering  disciplines,  but  the  distinct  differences  between  the  two  terms  are 
often  not  clearly  imderstood. 

•  Verification  determines  the  accuracy  and  consistency  of  software  algorithms 
compared  to  the  underlying  mathematical  models  [2].  A  verified  software 
program  should  be  algorithmically  correct  and  any  numerical  methods  used 
shotdd  lead  to  solutions  having  the  accuracy  required  by  the  application. 
Verification  is  often  associated  with  software  debugging  procedures. 

•  Validation  is  a  more  fundamental  concept  in  that  it  considers  the  relationship 
between  the  imderlying  mathematical  model  and  the  real  world  system  upon 
which  it  is  based  [2].  In  other  words,  validation  is  concerned  with  how  well  models 
represent  real  physical  processes. 

Ideally,  it  should  not  be  necessary  to  verify  commercial  software  since  it  is  normal  to 
assume  that  quality  assurance  practices  and  software  testing  procedures  in  a  commercial 
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establishment  are  sufficiently  comprehensive  to  ensure  reasonably  bug-free  programs. 
Consequently,  the  verification  of  SensorVision  is  not  considered  to  any  depth  within  this 
report  and  the  validation  of  SensorVision  is  the  main  focus. 

With  validation,  there  are  two  main  model  attributes  that  are  of  concern:  accuracy,  and 
realism.  Accuracy  refers  to  the  ability  of  a  model  to  correctly  characterise  some  physical 
process,  while  realism  additionally  requires  all  significant  and  relevant  physics  to  be 
modelled.  For  example,  a  model  may  characterise  thermal  emission  processes  accurately, 
but  if  reflection  phenomena  are  ignored,  the  model  may  be  not  be  realistic. 

In  general,  there  are  two  strategies^  that  can  be  adopted  in  a  validation  effort: 

•  Mathematical  validation  involves  the  comparison  of  data  obtained  from  the  model 
with  data  obtained  from  other  (non-real-time)  models  that  are  considered  accurate. 
Mathematical  validation  is  primarily  concerned  with  the  accuracy  of  the  models. 

•  Experimental  validation  involves  the  comparison  of  model  generated  results 
against  experimentally  measured  data.  Experimental  validation  may  also  consider 
the  realism  of  the  model  as  well  as  the  accuracy  aspects. 

The  SensorVision  validation  wiQ  be  conducted  from  both  mathematical  and 
experimental  perspectives.  However,  because  mathematical  validation  is  the  more 
economical  and  less  labour  intensive  option,  this  strategy  wiU  mainly  be  applied. 

1.3  Proposed  Validation  Procedures  for  HIL  Simulations 

The  purpose  and  aim  of  a  validation  program  are  often  difficult  to  articulate.  In  most 
cases,  models  are  not  perfect  and  there  are  difficult  questions  to  answer  in  relation  to 
how  results  from  a  validation  program  should  be  interpreted.  In  particular,  with  respect 
to  HIL  simulations  and  the  IRSG  fimction,  some  pertinent  questions  include: 

•  How  realistic  does  the  SensorVision  models  and  resultant  generated  unages  have 
to  be? 

•  What  are  the  effects  of  artifacts  in  the  virtual  environment  on  the  response  of  the 
seeker? 

These  questions  can  only  be  answered  by  experts  with  intimate  knowledge  of  the 
subsystems  of  the  seeker  being  tested,  given  sufficient  information.  With  this  truth  in 
mind,  a  two  tier  approach  to  the  validation  program  is  proposed. 

First,  the  validation  effort  should  be  conducted  with  a  view  to  develop  general 
guidelines  that  could  be  employed  during  the  simulation  construction  phase  of  a  FIIL 
procediue.  For  example,  simple  guidelines  could  be  used  to  identify  conditions  that  are 

Obviously,  both  validation  strategies  assuine  verified  models.  An  unverified  model  may  generate  incorrect 
results  due  to  errors  in  the  algorithm  code  thus  it  cannot  be  properly  validated. 
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likely  to  cause  erroneous  or  imrealistic  scenes.  These  conditions  could  then  be  avoided 
in  the  simulation. 

Second,  some  post-simulation  validation  procedure  should  be  developed  to  enable  users 
of  HIL  simulations  to  form  opinions  as  to  the  validity  of  a  set  of  HIL  results.  Ideally,  some 
measure  of  quality  should  be  provided  to  indicate  the  sorts  of  errors  associated  with  the 
simulation.  Fiurthermore,  it  would  be  advantageous  if  the  origins  of  the  errors  are  also 
provided.  The  most  promising  idea  is  to  formulate  a  procedxue  whereby  frame  grabs  are 
obtained  at  various  instances  during  a  simulation.  In  post-simulation  operations,  those 
same  frames  are  processed  again  using  accurate  non-real-time  models.  A  certain  amount 
of  confidence  needs  to  be  placed  in  the  ability  of  the  non-real-time  models  to  produce 
accurate  and  realistic  images.  An  error  frame  showing  radiometric  error  figures 
(between  the  SensorVision  generated  results  and  the  non-real-time  accurate  model) 
could  then  be  generated  and  presented  to  experts  for  evaluation. 

The  proposed  two  tier  approach  should  be  sufficient  for  realistic  HIL  simulations  to  be 
created  and  subsequently  evaluated. 

1.4  Report  Outline 

This  report  consists  of  two  distinct  sections.  Chapter  2  discusses  the  validation  effort 
associated  with  the  SensorVision  model  used  to  compute  the  emitted  radiance  from  an 
object  due  to  thermal  emission.  The  model  is  mathematically  validated  against  an 
acciuate  non-real-time  model,  and  a  meastue  of  quality  figure  is  developed.  This 
measure  of  quality  can  be  used  dining  the  simulation  construction  phase  to  rapidly 
assess  the  likely  level  of  errors  associated  with  that  particular  simulation.  In  addition, 
general  guidelines  are  developed  to  indicate  simulation  scenarios  that  should  be  avoided 
to  minimise  SensorVision  generated  errors. 

The  second  main  component  of  the  report,  which  discusses  the  validation  of  the 
temperature  prediction  model  used  by  SensorVision,  is  presented  in  Chapter  3.  The 
temperature  prediction  model  is  validated  against  experimental  data  obtained  from 
trials  conducted  in  Northern  Australia.  Measured  temperature  diurnal  profiles  are 
compared  against  SensorVision  generated  results  and  conclusions  are  formulated  based 
on  these  results. 

The  validation  described  in  Chapter  2  and  Chapter  3  provides  an  excellent  indication  of 
SensorVision's  ability  to  model  signature  components  due  to  thermal  emission 
processes.  The  realism  aspects  of  SensorVision  are  not  really  considered  in  this  report 
since  reflection  and  path  scattering  processes  are  not  validated.  Chapter  4  provides  a 
summary  of  conclusions  and  recommendations  for  further  work. 
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2.  Validation  of  the  Thermal  Emission  Model 

2.1  The  SensorVision  Thermal  Emission  Model 

Reference  [1]  describes  in  detail  the  thermal  emission  model  employed  by  SensorVision. 
The  model  is  an  equation  that  computes  the  radiance  received  by  a  ser\sor  due  to  thermal 
emission  processes  from  a  material  with  a  surface  temperature,  T.  The  models  used  to 
predict  the  surface  temperature  T  is  discussed  in  Chapter  3. 

The  SensorVision  thermal  emission  model  is  based  on  the  following  physics. 
Fimdamentally,  the  emitted  radiance^  from  any  solid  surface  is  described  in  terms  of  the 
thermal  emission  from  a  blackbody.  The  blackbody  is  an  idealised  material,  with 
properties  that  make  it  a  perfect  absorber  and  a  perfect  radiator.  All  energy  incident  on 
a  blackbody  is  completely  absorbed  and  consequently  radiated  in  order  to  maintain 
thermal  equilibrixim.  The  spectral  radiance  )  emitted  from  a  blackbody  of 
temperature  (T)  is  modelled  by  the  well  known  Planck's  blackbody  equation,  and  is 
given  by 


rbb 

- 


r  ^  \ 


rfAT  -2  -1  -K 

(Wm  sr  pm  ). 


nX' 


XT  1 
e  - 1 


The  parameters  Cj  and  C2  are  radiation  constants,  and  are  given  by 


Cj  =  3.7418x10^  (Wm  and 


C2  =  1.438769x10  (pmK),  respectively. 


(2-1) 


(2-2) 

(2-3) 


The  subscript  "X"  is  used  to  indicate  the  wavelength  dependence  of  the  blackbody 
spectral  radiance.  The  integration  of  (2-1)  over  waveband  of  interest  yields  the  radiance 
emitted  from  a  blackbody  within  that  waveband.  For  a  real  material,  a  property  referred 
to  as  the  spectral  emissivity  (6;^ )  represents  the  material's  efficiency  of  emission  relative 
to  a  blackbody  radiator.  Assuming  the  material  is  Lambertian^,  the  spectral  radiance 
emitted  by  the  material  is  given  by 

Radiance  is  the  fundamental  unit  of  infrared  physics.  Radiance  (or  sterance)  is  defined  as  the  power  per 
unit  projected  area  per  unit  solid  angle. 

A  Lambertian  material  is  one  in  which  the  radiance  emitted  by  the  material  is  independent  of  the 
observation  angle.  In  other  words,  a  Lambertian  material  is  an  isotropic  radiator  with  respect  to 
radiance  (not  intensity).  Lambertian  materials  are  also  often  described  as  diffuse. 
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■remitted  rbb 

h  =  ■ 


(2-4) 


If  the  material  is  opaque  so  that  transmission  processes  are  neglected  then  the 
relationship  between  spectral  emissivity  and  spectral  reflectivity^  (Px )  given  by 


=  1-Px- 


(2-5) 


Equation  (2-5)  is  simply  a  statement  of  the  conservation  of  energy.  Taking  into  account 
the  spectral  response  of  the  sensor"*  ((P;^^)  and  the  atmospheric  transmission  coefficient 
(T;^  )  along  the  LOS  (line  of  sight)  path  between  the  material  and  the  sensor,  the  spectral 
radiance  detected  by  the  sensor  is  given  by 

r  sensor  rbb 

^X  -  ^x'^xH^X  •  (2-6) 


The  radiance  detected  over  the  waveband  ^  j  to  ^-2  is  given  by 

(Wm-^sr-’).  (2-7) 

Equation  (2-7)  is  the  generally  accepted  physically  accurate  equation  for  calculating  the 
radiance  incident  on  a  sensor  due  to  thermal  emission  processes,  and  will  be  referred  to 
as  the  "exact"  model.  The  radiating  material  is  assumed  to  be  Lambertian,  opaque,  and 
solid.  Note  that  thermal  emissions  due  to  the  atmosphere  along  the  LOS  path  from  the 
material  to  the  sensor  are  not  modelled  by  (2-7).  SensorVision  separates  the  thermal 
emissions  due  to  the  atmospheric  path  from  the  emissions  due  to  solid  opaque 
materials  [1]. 


The  problem  with  employing  (2-7)  in  a  real-time  signature  prediction  code  is  that  the 
equation  involves  a  spectral  integration,  which  is  computationally  time  consuming  to 
evaluate.  Consequently,  SensorVision  uses  various  approximations  to  increase  the  speed 
with  which  the  thermal  emission  quantities  can  be  calculated.  The  SensorVision  thermal 
emission  model  is  based  on  the  equation 

Spectral  reflectivity  is  the  property  of  a  surface  that  specifies  the  amount  of  spectral  irradiance  (total 
spectral  radiance  incident  on  Ae  surface  area)  reflected  by  the  surface.  It  is  defined  as  the  ratio  of  the 
spectral  exitance  (total  spectral  radiance  reflected  by  total  surface  area)  to  the  spectral  irradiance. 

The  parameter  (cp;^^ )  is  used  by  SensorVision  to  crudely  model  any  attributes  of  the  sensor  that  causes 
frequency  selectivity  in  its  response.  The  parameter  may  include  the  lumped  effect  of  transmission 
through  the  sensor  optics,  detector  responsivity,  or  any  other  frequency  selective  sensor  property. 
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In  comparison  to  (2-7),  the  spectral  terms  and  have  been  moved  outside  of  the 
spectral  integration  and  replaced  by  their  in-band  averaged  values  t  and  £ .  The  bar 
notation  is  used  to  denote  the  averaging  (mean)  function.  The  integration  term  in  (2-8)  is 
essentially  precomputed  as  a  function  of  temperature,  and  stored  in  a  multivariate 
database  (or  lookup  table).  Consequently,  the  SensorVision  thermal  emission  model 
involves  only  fast  multiplication  calculations  so  that  it  can  be  executed  in  real-time.  The 
direct  comparison  of  (2-7)  to  (2-8)  constitutes  a  mathematical  validation.  The 
determination  of  whether  the  use  of  lookup  tables  is  accurate  is  a  verification  activity. 
Only  the  mathematical  validation  of  the  SensorVision  thermal  emission  model  will  be 
considered  in  this  report. 

The  manipulation  from  (2-7)  to  (2-8)  is  valid  only  if  the  spectral  atmospheric  transmission 
coefficient  and  the  material's  spectral  emissivity  are  constant  within  the  waveband  of 
interest.  Interestin|;ly,  the  approximation  is  also  valid  if  the  argument  of  the  spectral 
integration  ^  constant  within  the  waveband  of  interest.  The  proof  for  this 

statement  is  as  follows: 

Consider  the  following  substitution 


T  rbb 

-  ^x^x  • 

(2-9) 

With  this  substitution,  (2-8)  can  be  expressed  in  the  form 

=  xl^^Lxdk, 

(2-10) 

where  the  superscript  "SV"  indicates  that  the  equation  refers  to  the  SensorVision 
implemented  form  of  the  thermal  emission  equation.  If  has  a  constant  value  L 
within  the  waveband  of  interest,  then 

=  xlL^'^dk 

(2-lOa) 

=>  =  xlL(k2-'k{). 

(2-lOb) 

Now  consider  (2-7)  expressed  in  the  form 
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^exact  _  p 

T 

(2-11) 

IfL;, 

is  again  constant  within  the  waveband  of  interest,  then 

^exact  ^ 

(2-lla) 

=> 

=  Lj 

(A,2  -  Aj) 

(^2  —  Aj) 

(2-1 lb) 

j.  exact  ^  j 

■J^'TxEp^dA' 

(A2  -  Aj) 

(2-1  Ic) 

1 

1 

=> 

=  Lte 

(2-lld) 

=> 

^exact  ^ 

(2-lle) 

=> 

j^exact  j^SV 

(2-llf) 

Consequently,  it  can  be  concluded  that  the  SensorVision  thermal  emission 
equation  (2-8)  is  exactly  equivalent  to  the  accurate  equation  (2-7)  if  the  term 
is  constant  within  the  waveband  of  interest.  Note  that  the  step  from  (2-lld)  to 
(2-lle)  requires  the  variables  and  to  be  independent.  Given  that  the 
emissivity  is  a  material  property  and  the  transmission  coefficient  is  an  atmospheric 
property,  the  variables  and  are  clearly  independent. 

To  reiterate,  the  SensorVision  equation  is  equivalent  to  the  accurate  thermal  emission 
equation: 

•  if  the  spectral  atmospheric  transmission  coefficient  and  the  material's  spectral 
emissivity  are  constant  within  the  waveband  of  interest;  or 

•  if  the  argument  of  the  spectral  integration  is  constant  within  the  waveband 

of  interest. 

In  this  validation  effort,  it  is  assumed  that  the  spectral  response  of  the  sensor  is  unity  so 
that  the  argument  of  the  spectral  integration  is  simply  the  spectral  blackbody  radiance. 
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2.2  Figure  of  Merit  using  the  Second  Moment 

Mathematical  validation  of  the  SensorVision  thermal  emission  equation  essentially 
involves  the  determination  of  the  level  of  errors  associated  with  the  approximation 
described  in  Section  2.1.  In  this  approximation,  spectral  terms  are  moved  outside  of  the 
spectral  integration  and  replaced  by  their  in-band  averaged  values.  From  the  discussion 
in  Section  2.1,  the  level  of  errors  associated  with  the  SensorVision  thermal  ertussion 
model  is  clearly  dependent  on: 

•  the  variability  of  the  material's  spectral  emissivity  in  the  waveband  of  interest; 

•  the  variability  of  the  spectral  atmospheric  transmission  coefficient  in  the  waveband 
of  interest;  and 

•  the  variability  of  the  spectral  blackbody  radiance  in  the  waveband  of  interest 
(assuming  the  spectral  response  of  the  sensor  is  unity). 

In  Section  1.3,  a  two  tier  validation  strategy  for  HIL  simulations  was  proposed.  In  the  first 
tier  involving  the  use  of  gtudelines  for  simulation  construction,  some  figure  of  merit 
should  ideally  be  defined  to  enable  the  rapid  assessment  of  likely  error  levels  given 
details  of  the  simulation.  Since  the  error  levels  are  primarily  dependent  on  the  variability 
of  several  spectral  parameters,  the  figure  of  merit  should  logically  be  based  on  the  second 
moment  or  RMS  (root  mean  square)  error.  The  second  moment  is  a  value  often  used  in 
mathematics  to  express  the  variability  of  a  function  or  parameter.  The  second  moment  is 
defined  by  the  equation 


e  = 


(2-12) 


where  Ajc  is  the  difference  between  the  value  x-^  and  the  mean  value  x .  For  a  continuous 
variable,  the  second  moment  can  be  calciilated  using 


? 

(X;(^-jc)  dk 
\  _ 


i  (^2-^1)  ' 


(2-13) 


while  the  equation  relevant  for  a  discrete  variable  x^(n)  with  N  samples  is 


e 


N 

71=  1 


(2-14) 


During  the  SensorVision  validation  effort,  the  second  moment  will  be  used  to  develop 
general  guidelines  for  simulation  construction. 
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2.3  Effect  of  Variability  of  Material  Emissivity 

The  first  step  in  the  validation  effort  was  to  determine  the  effect  of  variability  in  the 
material's  spectral  emissivity  on  the  SensorVision  thermal  emission  model.  This  task  was 
accomplished  by  comparing  radiance  values  computed  using  (2-7)  and  (2-8)  for  a  range 
of  materials.  Two  wavebands  are  of  primary  concern:  the  MWIR  (mid-wave  mfrared) 
band;  and  the  LWIR  (long-wave  infrared)  band.  The  MWIR  band  nominally  spans  the 
waveband  from  3  pm  to  5  pm,  and  is  the  band  usually  exploited  in  air-to-air  missiles.  Hot 
exhaust  plumes  and  hot  engine  parts  tend  to  be  very  visible  in  the  MWIR  band.  The 
LWIR  band  extends  from  about  8  pm  to  12  pm,  and  is  the  band  often  used  in  the  imaging 
of  groimd  based  scenes.  Thermal  emission  processes  dominate  in  the  LWIR  band,  and  it 
is  the  waveband  in  which  thermal  differences  between  objects  with  temperatures  near 
ambient  (=  300  K)  can  be  most  efficiently  detected.  In  the  following  validation,  the 
atmospheric  transmission  coefficient  is  assumed  to  be  equal  to  one  and  a  material 
temperature  of  300  K  is  used.  Consequently,  the  modified  exact  and  SensorVision 
equations  are 


r  exact  rbb  t  ..i-x 

L  =  ^/Snd  (2-15) 

•'^1 

=  8 1  ,  respechvely.  (2-16) 

•’^1 

The  error  between  the  SensorVision  and  exact  models  for  a  range  of  materials  were 
computed,  and  are  plotted  in  the  bar  graphs  of  Figure  2-1.  The  errors  are  represented 
relative  to  the  exact  radiance  values  according  to 

jSV  j  exact 

error  =  ^  - .  (2-17) 

jjexact 

A  total  of  110  materials  were  tested  including  various  composites,  soils,  vegetation, 
construction  materials,  and  paints.  These  110  materials  represent  the  entire  materials 
database  packaged  with  SensorVision  version  2.2.  The  emissivity  (or  reflectivity)  data 
used  in  the  computations  were  the  default  data  for  the  materials  included  with  the 
SensorVision  package.  These  data  are  claimed  to  be  representative  of  typical  materials. 
The  materials  are  labelled  along  the  x-axis  of  the  bar  graphs  in  Figure  2-1  and  are 
identified  by  their  material  codes.  Table  2-1  contains  the  material  descriptions  with  their 
codes. 
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Figure  2-1  (a)  shows  the  SensorVision  errors  associated  with  the  MWIR  band.  As  can  be 
seen,  the  errors  fluctuate  significantly  depending  on  the  material.  A  relative  error  of 
22.7  %  is  associated  with  the  material  oxidised  aluminium.  In  comparison,  the  errors 
shown  in  Figure  2-l(b)  for  the  LWIR  band  are  negligible.  These  errors  are  only  a  fraction 
of  a  percent  for  all  materials  tested.  Despite  these  results,  it  is  not  prudent  to  immediately 
conclude  that  SensorVision  is  ill-smted  to  the  MWIR  band  and  well-stuted  to  the  LWIR 
band.  In  fact,  these  results  have  a  temperature  dependence  and  the  low  errors  associated 
with  the  LWIR  band  is  mainly  due  to  the  chosen  temperature  of  300  K.  The  temperature 
dependence  will  be  explained  in  greater  detail  in  Section  2.5.  For  now,  it  suffices  to  state 
that  no  definite  conclusions  can  be  drawn  from  the  results  plotted  in  Figure  2-1,  besides 


Figure  2-1:  Bar  graphs  showing  the  relative  error  caused  by  the  SensorVision  thermal  emission 
model  as  a  function  of  material  for  (a)  the  MWIR  band  and  (b)  the  LWIR  band.  The  temperature 
is  assumed  to  be  300  K  and  the  atmospheric  transmission  coefficient  is  assigned  a  value  of  unity. 
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Code 

Material 

Code 

Material 

Code 

Material 

COMPOSITES 

SOILS 

VEGETATION 

1 

beige_fabric 

46 

beach_sand 

83 

broadleaf 

2 

black_rabber 

47 

black_sand 

84 

broadleaf  (shaded) 

3 

brown_plastic 

48 

clay_soil 

85 

broadleaf-pine 

4 

cream_fiberglass 

49 

compact_soil 

86 

broadleaf-pine  (shaded) 

5 

gold_nylon 

50 

desert_sand 

87 

broadleaf-scrub 

6 

green_canvas 

51 

dryjakebed 

88 

broadleaf-scrub  (shaded) 

7 

grey_fabric 

52 

dry_seabed 

89 

dry_grass 

8 

olive_plastic 

53 

fresh_snow 

90 

diy_grass  (shaded) 

9 

orange_epoxy 

54 

ice 

91 

grass-scrub 

10 

red_nylon 

55 

lake_sand 

92 

grass-scrub  (shaded). 

11 

tan_felt 

56 

limestone-silt 

93 

grass-soil 

12 

white_fabric 

57 

limestone-silt-sand 

94 

grass-soil  (shaded) 

13 

yellow_silicon 

58 

limestone-silt-soil 

95 

lawn_grass 

14 

asphalt_shingles 

59 

limestone_rock 

96 

lawn_grass  (shaded) 

60 

loam_soil 

97 

pine 

CONSTRUCTION 

61 

loamy_sand 

98 

pine  (shaded) 

15 

black_asphalt 

62 

lump_coal 

99 

pine-broadleaf 

16 

block_concrete 

63 

old_snow 

100 

pine-broadleaf  (shaded) 

17 

clear_glass 

64 

road_gravel 

101 

pine-scmb 

18 

dark_titanium 

65 

salt-silt 

102 

pine-scrub  (shaded) 

19 

meta]_roof 

66 

sand-soil 

103 

scrub 

20 

oxydized_aluminum 

67 

sandstone-soil 

104 

scmb  (shaded) 

21 

paved_concrete 

68 

sandstone_rock 

105 

scrub- grass-soil 

22 

pebbled_asphalt 

69 

sandy  _loam 

106 

scrub-grass-soil  (shaded) 

23 

pine_wood 

70 

scrub-soil 

107 

tree_bark 

24 

red_brick 

71 

scrub-soil  (shaded) 

108 

tree_bark  (shaded) 

25 

roof_tar 

72 

silt-sand 

109 

tundra 

26 

rusting_steel 

73 

silty_clay 

110 

hmdra  (shaded) 

27 

urban_commercial 

74 

siltyjoam 

28 

urban_residential 

75 

tilled_soil 

29 

wood_siding 

76 

vamished_sand 

77 

vamished_sandstone 

PAINTS 

78 

water 

30 

ash_grey 

79 

wetjakebed 

31 

battleship_grey 

80 

wet_seabed 

32 

blackish_brown 

81 

wet_soil 

33 

bleached_yellow 

82 

white_sand 

34 

canary  _yellow 

35 

dark_tan 

36 

jet_black 

37 

kelly  _green 

38 

light_red 

39 

light_tan 

40 

niint_green 

41 

mustard_yellow 

42 

off_white 

43 

olive_green 

44 

sky_blue 

45 

snow_white 

Table  2-1:  List  of  materials  tested  in  validation  and  associated  material  codes. 
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the  fact  that  the  level  of  errors  associated  with  the  SensorVision  thermal  emission  model 
is  highly  dependent  on  the  material  emissivity. 

Figure  2-2  shows  the  same  results  as  those  displayed  in  Figure  2-1  except  that  they  have 
been  transformed  into  the  temperature  domain.  The  exact  and  SensorVision  computed 
radiance  values  are  converted  into  effective  temperatures,  which  are  more  intuitive  to 
interpret.  The  effective  temperature  is  defined  as  the  temperature  of  a  blackbody 
that  wotdd  generate  the  same  radiance  (L )  as  that  observed,  as  given  by 


¥ 

3 

RS 

O 

o. 

E 


w 

o 

lx 

W 
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Figure  2-2:  Bar  graphs  showing  the  error  in  effective  temperature  caused  by  the 
SensorVision  thermal  emission  model  as  a  function  of  material  for  (a)  the  MWIR  band 
and  (b)  the  LWIR  band.  The  temperature  is  assumed  to  be  300  K  and  the  atmospheric 
transmission  coefficient  is  assigned  a  value  of  unity. 
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(2-18) 


Function  Ly^  (T)  is  the  blackbody  equation  (2-1).  The  errors  in  Figxure  2-2  are  expressed 
in  terms  of  the  difference  in  effective  temperatures  between  the  SensorVision  generated 
results  and  the  exact  results. 


It  is  important  to  imderstand  that  the  SensorVision  errors  (expressed  as  percentage 
radiance  error  or  effective  temperature  difference)  due  to  the  variability  in  the  spectral 
emissivity  data  cannot  be  separated  from  the  effect  of  variations  in  Planck's  blackbody 
curve.  Consequently,  for  greater  insight,  the  second  moment  of  the  spectral  emissivity 
data  needs  to  be  computed.  Figure  2-3  shows  the  second  moment  values  for  the  spectral 


Figure  2-3:  Bar  graphs  of  second  moments  of  material  emissivities  for  (a)  the  MWIR  band,  and 

(b)  the  LWIR  band. 
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emissivity  data  for  the  110  materials  considered.  A  second  moment  of  zero  corresponds 
to  the  best  case  where  the  emissivity  is  constant  over  the  waveband,  while  a  second 
moment  of  0.5  corresponds  to  the  worst  case.  In  the  worst  case,  the  material's  spectral 
emissivity  has  a  value  of  one  for  exactly  half  of  the  waveband  and  zero  for  the  other  half. 
Overlaid  on  Figure  2-3(a)  and  Figure  2-3(b)  are  the  relative  error  results  from  Figure 
2-l(a)  and  Figure  2-l(b),  respectively.  These  errors  are  normalised  to  enable  a  visual 
comparison  with  the  second  moment  values,  hence  they  are  not  associated  with  any 
y-axis  scale.  As  can  be  observed  in  the  bar  graphs  of  Figure  2-3,  the  second  moment  does 
indeed  provide  a  reasonable  method  of  estimating  the  level  of  errors  associated  with  the 
use  of  certain  materials  in  SensorVision  simulations.  Fxurthermore,  comparisons  of 
Figure  2-3(a)  and  Figure  2-3(b)  indicate  that  the  material  emissivity  data  tend  to  fluctuate 
less  in  the  LWIR  band  compared  to  the  MWIR  band.  This  inference  is  further  supported 
by  an  average  second  moment  value  of  0.0159  in  the  LWIR  band  compared  to  an  average 
value  of  0.0349  in  the  MWIR  band.  Note  that  the  magnitude  of  improvement  in  the 
second  moment  value  for  the  LWIR  band  compared  to  the  MWER  band  does  not  directly 
reflect  the  level  of  improvement  in  the  relative  errors  (of  Figure  2-1)  or  the  effective 
temperature  difference  values  (of  Figmre  2-2).  The  second  moments  computed  in 
Figure  2-3  is  dependent  only  on  the  spectral  emissivity  data,  while  the  relative  error  (or 
the  effective  temperature  difference)  depends  on  both  the  spectral  emissivity  and  the 
temperature.  The  relative  error  calculations,  are  biassed  by  variations  in  the  Planck's 
blackbody  curve  (see  Section  2.5),  and  are  therefore  not  good  independent  indicators  of 
which  materials  cause  SensorVision  errors. 

From  the  results  of  this  validation,  it  is  concluded  that  the  SensorVision  thermal  emission 
model  is  more  suited  to  LWIR  simulations  since  materials  tend  to  have  spectral 
emissivity  curves  that  fluctuate  less  in  the  8  to  12  jrm  band  compared  to  the  3  to  5  pm 
band.  Given  that  the  second  moment  has  a  maximum  value  of  0.5  and  a  minimum  value 
of  zero  for  emissivity  data,  a  reasonable  figure  of  merit  or  error  indicator  (El)  with  a  more 
intuitive  range  of  zero  to  100  can  be  defined  as 

N 

EI^  =  100  X  2e^  =  100  X  ^  [E;^(n)  -  ef .  (2-19) 

n  =  l 


Table  2-2  lists  the  EI^  values  for  the  110  materials  currently  in  the  SensorVision  materials 
database.  This  table  can  be  consulted  when  constructing  SensorVision  simulations  in 
order  to  make  competent  decisions  on  whether  certain  materials  should  be  included  in 
the  simulation. 
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Error  Indicator 

(£4) 


MWIR  LWm 


Error  Indicator 

(.El,) 


MWIR  LWIR 


Error  Indicator 

(Eh) 


MWIR  LWIR 


Table  2-2:  Emissivity  error  indicator  values  for  materials. 
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2.4  Effect  of  Variability  of  Atmospheric  Transmission  Coefficient 

In  the  validation  described  in  Section  2.3,  the  atmospheric  transmission  coefficient  was 
assumed  to  have  a  constant  value  of  one  and  it  was  subsequently  ignored.  In  this  section, 
the  effect  of  the  spectral  atmospheric  transmission  coefficient  on  the  accuracy  of  the 
SensorVision  thermal  emission  model  will  be  assessed.  The  validation  procedure  used  in 
this  section  is  essentially  the  same  as  that  employed  in  the  previous  section.  The 
variability  of  the  spectral  transmission  coefficient  can  be  quantified  by  using  the  second 
moment,  and  an  error  indicator  (El)  can  be  defined  according  to 

N 

£7,  =  100x|5;  [Ti(n)-ti.  (2-20) 

n=  1 


Again,  an  El  value  of  zero  represents  the  best  case  while  a  value  of  100  indicates  the  worst 
case. 

The  atmospheric  transmission  coefficient  is  a  complex  parameter  that  is  dependent  on 
several  factors  including  the  weather  conditions,  the  atmospheric  profile,  the  waveband, 
and  the  LOS  path  between  the  sensor  and  the  radiating  material.  It  is  a  difficult 
parameter  to  predict  due  to  its  dependence  on  many  influences.  In  this  validation  effort, 
a  program  called  the  MOSART  (Moderate  Spectral  Atmospheric  Radiance  and 
Transmittance)  code  is  used  to  generate  the  spectral  atmospheric  transmission  coefficient 
for  a  range  of  different  LOS  paths  for  both  the  MWIR  and  LWIR  bands.  The  transmission 
coefficients  were  computed  for  a  location  in  northern  Australia  for  the  date  31st  January 
1998  at  time  15:00  LST  (local  standard  time)  or  5:30  GMT  (Greenwich  Mean  Time).  The 
choice  of  these  parameters  was  mainly  arbitrary.  The  LOS  paths  for  which  the  spectral 
transmission  coefficients  were  computed  are  listed  in  Table  2-3,  and  included  horizontal 
paths  to  90°  slant  paths  with  ranges  varying  from  1  km  to  50  km.  In  all  cases,  the  altitude 
at  the  lower  end  of  the  path  was  100  m.  The  computed  El  values  associated  with  the 
transmission  coefficients  for  these  paths  are  listed  in  Table  2-4.  From  Table  2-4,  it  is  clear 
that  the  El  values  are  typically  smaller  in  the  LWIR  band  compared  to  the  MWIR  band. 
In  fact,  some  of  the  El  values  for  the  MWER  band  are  very  large  (with  values  in  excess  of 
70  compared  to  a  worst  case  value  of  100)  and  indicate  that  the  transmission  coefficient 
varies  significantly  in  the  MWIR  band.  Scrutiny  of  Figure  2-4,  which  contains  some 
atmospheric  traiismission  coefficient  plots,  reveals  the  reasons  for  these  observations. 
The  variability  of  the  transmission  coefficient  curve  is  clearly  lower  in  the  LWIR  band 
compared  to  the  MWIR  band.  In  particular,  there  is  a  complete  null  at  4.3  pm  due  to  CO2 
(carbon  dioxide)  absorption,  which  contributes  to  a  large  El  value  in  the  MWIR  band. 
The  general  decrease  in  El  values  as  the  range  increases  is  caused  by  the  fact  that  the 
atmospheric  attenuation  increases  with  range.  Consequently,  the  transmission 
coefficient  is  reduced  and  the  variability  in  the  spectral  curve  decreases.  Figure  2-4(b) 
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shows  a  transmission  coefficient  plot  for  a  high  attenuation  case  resulting  in  lower 
variability  in  both  the  MWIR  and  LWIR  cases,  compared  to  Figure  2-4(a).  As  the 
elevation  angle  for  the  LOS  path  approaches  90  degrees,  the  effect  of  range  becomes  less 
pronounced  due  to  the  thinner  atmosphere  at  higher  altitudes.  Furthermore,  in  the  LWIR 
case,  the  introduction  of  some  high  altitude  atmospheric  molecular  species  has  caused 
additional  attenuation  near  10  pm,  which  was  not  evident  at  lower  altitudes  (compare 
Figure  2-4(d)  to  Figure  2-4(c)).  This  extra  attenuation  at  10  pm  increases  the  variability  of 
the  LWIR  transmission  coefficient  and  hence  causes  a  slight  increase  in  the  El  values  as 
the  LOS  path  extends  to  higher  altitudes. 

The  main  conclusion  drawn  from  the  results  contained  in  Table  2-4  is  that  the 
SensorVision  thermal  emission  model  is  more  suited  to  LWIR  simulations  than  MWIR 
simulations.  The  large  variations  in  the  spectral  atmospheric  traitsmission  coefficient  in 
the  MWIR  band  may  introduce  significant  errors  in  the  SensorVision  simulation. 
However,  the  level  of  errors  will  reduce  if  narrower  bands  within  the  MWIR  band  are 
considered,  especially  if  the  null  in  the  vicinity  of  4.3  pm  is  omitted  from  the  waveband. 

In  a  HIL  simulation,  it  is  difficult  to  determine  what  type  of  LOS  paths  are  likely  to  be 
encoimtered  within  the  simulation.  However,  given  that  the  engagement  scenario  is 
known  prior  to  the  simulation,  the  results  contained  within  Table  2-4  will  aid  in  the 
determination  of  the  likely  level  of  errors  associated  with  different  parts  of  the 


Code 

Path  Description 

Range 

Code 

Path  Description 
— . . . . 

Range 

1-1 

Horizontal  Path 

1  km 

4-1 

Slant  Path  30°  Elevation 

1  km 

1-2 

2  km 

4-2 

2  km 

1-3 

5  km 

4-3 

5  km 

1-4 

10  km 

4-4 

10  km 

1-5 

20  km 

4-5 

20  km 

1-6 

50  km 

4-6 

50  km 

2-1 

Slant  Path  10°  Elevation 

1  km 

5-1 

Slant  Path  60°  Elevation 

1  km 

2-2 

2  km 

5-2 

2  km 

2-3 

5  km 

5-3 

5  km 

2-4 

10km 

5-4 

10  km 

2-5 

20  km 

5-5 

20  km 

2-6 

50  km 

5-6 

50  km 

3-1 

Slant  Path  20°  Elevation 

1  km 

6-1 

Slant  Path  90°  Elevation 

1  km 

3-2 

2  km 

6-2 

2  km 

3-3 

5  km 

6-3 

5  km 

3-4 

10  km 

6-4 

10  km 

3-5 

20  km 

6-5 

20  km 

3-6 

50  km 

6-6 

50  km 

Table  2-3;  List  of  LOS  paths  tested  in  validation  of  SensorVision  thermal  emission  model. 
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Table  2-4:  Error  indicator  values  for  LOS  paths  tested. 
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engagement.  For  example,  consider  an  air  to  ground  scenario  where  a  weapon  is 
laimched  from  an  air  platform  at  a  range  of  10  km.  Say  the  weapon  has  a  MWIR  seeker 
and  the  engagement  is  such  that  the  trajectory  of  the  missile  follows  an  approximately 
straight  path  with  an  elevation  of  approximately  10°.  From  the  El  values  listed  in  Table 
2-4,  it  is  expected  that  SensorVision  generated  scenes  will  have  errors  associated  with  the 
thermal  emission  model  that  are  moderate  at  the  start  of  the  engagement  (El  value  of 
around  49).  These  errors  become  progressively  worse  (El  value  approaching  71)  as  the 
missile  homes  in  on  the  target.  Actual  errors  will  also  depend  on  the  material 
composition  and  temperature  of  the  target. 

Having  analysed  the  variability  of  material  emissivities  and  atmospheric  transmission 
coefficients  independently,  the  next  question  that  needs  to  be  answered  is  the  effect  of 
both  parameters  on  the  SensorVision  thermal  emission  model.  In  essence,  it  is  necessary 
to  determine  the  variability  of  the  product  of  the  emissivity  and  transmission  coefficient 
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Figure  2-4:  Plot  of  atmospheric  transmission  coefficient  as  a  function  of  wavelength 
for  a  selection  of  LOS  paths:  (a)  a  horizontal  1  km  path;  (b)  a  horizontal  10  km  path; 
(c)  a  vertical  1  km  path;  and  (d)  a  vertical  50  km  path. 
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parameters.  A  simple  and  quick  method  of  combining  the  second  moments  of  the 
emissivity  and  transmission  coefficient  is  to  apply 


(2-21) 


Parameter  is  the  second  moment  value  of  the  emissivity  and  transmission  coefficient 
product,  and  f  are  the  second  moment  and  mean  of  the  transmission  coefficient, 
respectively,  and  and  S  are  the  second  moment  and  mean  of  the  emissivity, 
respectively.  Equation  (2-21)  is  a  formula  used  to  calculate  errors  in  measurements  due 
to  multiplication  operations.  It  is  only  an  approximate  formula  but  it  should  be 
sufficiently  accurate  for  establishing  guidelines  in  the  SensorVision  validation.  The 
application  of  (2-21)  is  easier  than  directly  computing  the  second  moment  of  the 
emissivity  and  transmission  coefficient  product.  An  appropriate  error  indicator  for  the 
product  can  be  defined  as 

=  100x2e„.  (2-22) 

The  error  between  the  exact  equation  and  the  SeiasorVision  thermal  emission  model  was 
calculated  for  all  110  materials  listed  in  Table  2-1  and  the  36  LOS  paths  listed  in  Table  2-3. 
A  surface  temperature  of  300  K  was  again  assumed,  and  both  the  MWIR  and  LWIR 
bands  were  considered.  The  resultant  data  set  contained  7920  values,  which  is  too  much 
information  to  present  in  this  report.  Table  2-5  siurunarises  the  results  by  displaying  the 
statistics  of  the  data  set  for  each  LOS  path.  As  evident  in  these  results,  errors  associated 
with  the  MWIR  band  are  significant.  Average  relative  errors  are  12.5%  for  a  range  of 
1  km  and  increase  to  a  value  of  83.8%  for  a  range  of  50  km  assuming  a  horizontal  LOS 
path.  Note  that  the  large  value  of  83.8%  for  the  50  km  range  is  caused  by  the  effect  of  the 
transmission  coefficient  on  the  calcixlated  radiances.  High  attenuation  causes  the  exact 
radiance  to  be  small  and  the  calculated  relative  error  is  amplified  due  to  the  quotient 
operation  in  (2-17).  For  this  reason,  the  relative  error  values  can  be  misleading  and  the 
errors  expressed  as  effective  temperatures  may  provide  a  better  indication  of  the 
inaccuracies  associated  with  the  SensorVision  thermal  emission  model.  Again,  it  is 
emphasised  that  the  actual  values  contained  in  Table  2-5  should  only  be  considered, 
keeping  in  mind  that  a  surface  temperature  of  300  K  has  been  assumed.  As  will  be 
explained  in  the  next  section,  the  chosen  temperature  of  300  K  amplifies  any  errors 
associated  with  the  MWIR  band  while  reducing  errors  associated  with  the  LWIR  band. 
The  El  values  contained  in  Table  2-2  and  Table  2-4  are  more  reliable  in  the  formulation 
of  conclusions  than  the  results  listed  in  Table  2-5.  Fiuthermore,  using  El  values  is  much 
easier  and  quicker  than  performing  the  laborious  task  of  calculating  relative  errors  or 
effective  temperature  differences. 
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LWIR  Band 

MWIR  Band 

Path 

Minimum 

Maximum 

Average 

Minimum 

Maximum 

Average 

Code 

AL 

AT 

AL 

AT 

AL 

AT 

AL 

AT 

AL 

AT 

AL 

AT 

(%) 

(K) 

(%) 

(K) 

(%) 

(K) 

(%) 

(K) 

(%) 

(K) 

(%) 

(K) 

1-1 

0.7 

0.2 

33.0 

5.5 

12.5 

2.8 

-1.5 

-0.5 

0.7 

0.3 

-03 

-0.2 

1-2 

12.3 

2.6 

40.5 

6.3 

21.1 

4.3 

-2.5 

-0.7 

1.6 

0.7 

-0.5 

-0.2 

1-3 

33.3 

6.1 

55.9 

8.0 

39.1 

7.0 

-5.1 

-0.9 

4.5 

1.4 

-0.9 

-03 

1-4 

48.7 

7.4 

70.1 

10.1 

56.2 

8.7 

-8.0 

-0.7 

9.0 

1.4 

-1.2 

-03 

1-5 

62.3 

7.8 

95.6 

11.2 

723 

9.4 

-10.5 

-0.1 

15.6 

0.2 

-13 

-0.0 

1-6 

71.1 

5.4 

118 

10.3 

83.8 

83 

-12.5 

-0.0 

26.7 

0.0 

-0.7 

-0.0 

2-1 

0.2 

0.1 

32.7 

5.5 

12.1 

2.7 

-1.4 

-0.4 

0.6 

0.3 

-03 

-03 

2-2 

10.6 

2.3 

39.5 

6.2 

19.8 

4.1 

-2.3 

-0.7 

1.3 

0.6 

-0.5 

-03 

2-3 

29.3 

5.4 

52.2 

7.4 

34.6 

6.4 

-4.2 

-0.9 

3.3 

1.1 

-0.8 

-03 

2-4 

40.7 

6.8 

62.8 

9.0 

47.2 

7.8 

-5.9 

-0.8 

5.5 

1.5 

-1.1 

-03 

2-5 

50.3 

7.5 

71.4 

10.1 

57.4 

8.8 

-6.8 

-0.8 

7.0 

1.5 

-1.1 

-03 

2-6 

58.8 

8.0 

81.0 

10.9 

66.1 

9.5 

-6.6 

-0.7 

7.4 

1.4 

-1.1 

-0.2 

3-1 

-0.2 

-0.1 

32.4 

5.4 

11.8 

2.6 

-1.3 

-0.4 

0.6 

0.3 

-0.3 

-0.1 

3-2 

9.1 

2.0 

38.6 

6.1 

18.8 

3.9 

-2.1 

-0.6 

1.2 

0.5 

-0.5 

-0.2 

3-3 

25.3 

4.7 

49.3 

7.2 

31.2 

5.9 

-3.5 

-0.9 

2.5 

0.9 

-0.7 

-03 

3-4 

33.7 

6.1 

56.3 

7.9 

393 

6.9 

-4.2 

-0.9 

3.3 

l.I 

-0.8 

-03 

3-5 

38.5 

6.6 

60.8 

8.5 

443 

7.5 

-4.2 

-0.8 

3.4 

1.1 

-0.8 

-03 

3-6 

41.3 

6.8 

63.4 

8.8 

47.1 

7.8 

-3.8 

-0.7 

3.3 

1.0 

-0.7 

-0.2 

4-1 

-0.6 

-0.1 

32.1 

5.4 

11.5 

2.6 

-1.3 

-0.4 

0.5 

0.3 

-0.3 

-0.1 

4-2 

7.9 

1.7 

37.8 

6.0 

17.9 

3.8 

-2.0 

-0.6 

1.0 

0.5 

-0.4 

-0.2 

4-3 

21.4 

4.1 

46.8 

6.9 

28.2 

5.5 

-3.0 

-0.8 

2.0 

0.8 

-0.6 

-03 

4-4 

27.6 

5.1 

51.4 

7.3 

333 

6.2 

-3.1 

-0.8 

2.1 

0.8 

-0.7 

-0.3 

4-5 

31.1 

5.5 

54.1 

7.6 

36.2 

6.6 

-3.0 

-0.7 

2.1 

0.8 

-0.6 

-0.2 

4-6 

33.3 

5.9 

56.1 

7.7 

38.5 

6.9 

-2.3 

-0.5 

2.0 

0.7 

-0.4 

-0.2 

5-1 

-1.5 

-0.3 

31.6 

5.3 

10.8 

2.4 

-1.2 

-0.4 

0.5 

0.2 

-03 

-0.1 

5-2 

5.5 

1.2 

36.3 

5.9 

16.1 

3.5 

-1.7 

-0.5 

0.8 

0.4 

-0.4 

-0.2 

5-3 

13.5 

2.8 

41.7 

6.5 

22.2 

4.5 

-2.0 

-0.6 

1.1 

0.5 

-0.5 

-0.2 

5-4 

16.6 

3.3 

44.1 

6.7 

24.7 

4.9 

-2.0 

-0.6 

1.0 

0.4 

-0.4 

-0.2 

5-5 

17.7 

3.5 

45.1 

6.8 

25.7 

5.1 

-1.8 

-0.5 

0.9 

0.4 

-0.4 

-0.2 

5-6 

19.7 

3.8 

46.6 

7.0 

27.4 

5.4 

-1.2 

-0.3 

0.8 

0.3 

-0.2 

-0.1 

6-1 

-1.7 

-0.4 

31.4 

5.3 

10.6 

2.4 

-1.2 

-0.4 

0.4 

0.2 

-03 

-0.1 

6-2 

4.8 

1.1 

35.8 

5.8 

15.6 

3.4 

-1.6 

-0.5 

0.8 

0.4 

-0.4 

-03 

6-3 

11.3 

2.4 

40.4 

6.3 

20.6 

43 

-1.8 

-0.5 

0.9 

0.4 

-0.4 

-0.2 

6-4 

13.7 

2.8 

42.4 

6.5 

22.7 

4.6 

-1.7 

-0.5 

0.8 

0.4 

-0.4 

-0.2 

6-5 

14.7 

3.0 

43.1 

6.6 

23.5 

4.7 

-1.5 

-0.4 

0.7 

0.3 

-03 

-0.2 

6-6 

16.3 

3.3 

44.4 

6.7 

24.8 

5.0 

-1.0 

-0.3 

0.6 

0.3 

-0.2 

-0.1 

Table  2-5:  Summary  of  errors  generated  by  SensorVision  thermal  emission  model  including 
both  material  emissivity  and  atmospheric  transmission  coefficient  data.  Errors  are  represented 
in  terms  of  percentage  error  (AL)  relative  to  the  exact  radiance,  and  the  difference  (AT)  between 
SensorVision  and  exact  effective  temperatures. 
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2.5  Effect  of  Variability  of  Planck's  Blackbody  Curve 

Planck's  blackbody  equation  (given  by  (2-1))  is  a  weU  known  function,  which  has  a 
temperahure  dependence.  Figure  2-5  contains  plots  of  Planck's  curves  as  functions  of 
temperature.  As  evident  in  these  plots,  the  peak  of  Planck's  curve  moves  to  the  shorter 
wavelengths  and  increases  in  magnitude  as  the  temperature  is  raised. 

The  variability  of  Planck's  blackbody  curve  within  the  waveband  of  interest  has  an 
impact  on  the  errors  associated  with  the  SensorVision  thermal  emission  model.  Even 
though  the  variations  in  the  emissivity  and  transmission  coefficient  parameters  are  the 
actual  soxurce  of  errors,  the  variability  of  Planck's  curve  substantially  affects  the  error 
magnitudes.  This  statement  is  supported  by  the  plots  shown  in  Figture  2-6.  In  Figure  2-6, 
the  mean  relative  error  of  the  SensorVision  thermal  emission  model  is  plotted  as  a 
function  of  temperature.  The  mecm  relative  error  is  computed  by  calculating  (2-17)  for  the 
110  materials  in  the  materials  database,  applying  a  modulus  function  to  remove  any  sign 
dependence,  and  determining  the  mean  over  all  materials.  Note  that  the  atmospheric 
transmission  coefficient  is  assumed  to  have  a  value  of  one  to  simplify  the  calculations. 

Figure  2-7  contains  a  plot  of  the  El  values  associated  with  the  variability  of  Planck's 
blackbody  curve  in  the  waveband  of  interest.  The  El  value  is  calculated  using 


Figure  2-5:  Plot  of  Planck's  blackbody  curves  as  a  function  of  temperature. 
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100x2 


-Ln^)  dk 


(2-23) 


where  Ln-^  is  the  Planck's  blackbody  curve  normalised  by  its  maximum  value  in  the 
waveband  of  interest.  Normalisation  is  required  to  Umit  Ln-^  to  values  between  zero 
and  one,  and  erasure  the  El  value  remains  in  the  range  zero  to  100. 
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Figure  2-6:  Mean  relative  error  between  SensorVision  thermal  emission  model  and  exact  model 
as  a  function  of  temperature.  Atmospheric  transmission  coefficient  is  assumed  to  be  unity  and 
all  110  materials  are  included  in  the  computation  of  the  mean  relative  error. 


Figure  2-7:  Plot  of  error  indicator  value  of  Planck's  blackbody  curve  in  waveband  of  interest  as 

a  function  of  temperature. 
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The  key  feature  to  note  in  Figure  2-6  and  Figure  2-7  is  the  similarity  between  the  curves. 
Clearly,  the  variability  of  the  Planck's  curve  in  the  waveband  of  interest  (quantified  by 
the  El  value)  has  an  effect  on  the  SensorVision  errors.  Furthermore,  the  effect  appears  to 
be  linear  m  nature  (at  least  in  the  temperature  range  300  K  to  1000  K).  The  minimum 
value  of  the  El  parameter  in  the  MWIR  and  LWIR  bands  correspond  to  the  temperatures 
of  740.2  K  and  293.7  K,  respectively.  This  result  is  significant  because  materials  used  in 
simulations  tend  to  have  temperatures  near  ambient  air  temperature,  which  is  near  the 
optimal^  temperature  for  the  LWIR  band.  Again,  it  appears  that  the  SeitsorVision 
thermal  emission  model  is  more  suited  to  LWIR  simulations.  However,  simulations 
involving  hot  bodies  (such  as  hot  plumes  and  hot  engine  parts  in  air  engagements)  may 
result  in  greater  errors  in  the  LWIR  band  compared  to  the  MWIR  band.  For  instance,  at 
a  temperature  of  700  K,  the  El  value  in  the  MWIR  band  is  12.6  compared  to  37.3  in  the 
LWIR  band.  The  mean  relative  error  at  700  K  (as  plotted  in  Figure  2-6)  is  0.2%  in  the 
MWIR  band  compared  to  0.46%  in  the  LWIR  band. 

Due  to  the  apparent  linear  relationship  between  the  SeiisorVision  thermal  emission 
errors  and  the  El  values  for  the  Planck's  blackbody  curves,  a  total  error  indicator  value 
can  be  estimated  by  applying 


El 


El, 


total 


100 


(2-24) 


The  error  indicator  value  El^,^  is  given  by  (2-22)  and  El is  given  by  (2-23).  The  total  El 
value  is  again  limited  to  the  range  zero  to  100,  where  a  value  of  100  corresponds  to  the 
worst  possible  case.  Again,  the  concept  behind  the  El  value  is  to  aid  users  in  the 
construction  of  SensorVision  simulations  for  ML  testing. 


2.6  Chapter  Summary 

In  this  chapter,  the  SensorVision  thermal  emission  model  (as  applied  to  solid  opaque 
Lambertian  scene  materials)  was  validated.  The  validation  involved  determining  the 
level  of  errors  associated  with  tiie  application  of  the  SensorVision  approximation 
whereby  spectral  terms  are  moved  outside  of  the  spectral  integration  and  replaced  by 
their  in-band  averaged  values.  The  two  spectral  terms  affected  were  the  material's 
spectral  emissivity  and  the  spectral  atmospheric  transmission  coefficient.  The  effect  of 
moving  each  of  these  parameters  outside  of  the  integration  was  determined 
independently  and  a  figure  of  merit  referred  to  as  the  error  indicator  (El)  was  proposed. 
The  development  of  the  El  was  one  of  Ihe  significant  outcomes  of  this  validation  effort. 
The  total  combined  El  value,  given  by  (2-4),  strongly  reflects  the  error  magnitude 

The  temperature  is  optimal  in  the  sense  that  it  minimises  the  errors  associated  with  the  Ser\sorVision 
thermal  emission  model. 
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associated  with  the  SensorVision  thermal  emission  model,  and  it  can  be  determined 
much  quicker  than  the  direct  computation  of  the  actual  error.  The  error  indicator  should 
be  used  in  the  construction  phase  of  a  HIL  simulation  to  identify  situations  and  scenarios 
whereby  the  performance  of  SensorVision,  in  producing  thermal  emission  signature 
components,  may  be  deficient. 

It  was  concluded  from  the  validation  effort  that  the  SensorVision  thermal  emission 
model  is  more  suited  to  LWIR  simulations.  The  reasons  for  this  conclusion  are  three-fold: 

•  spectral  emissivities  of  many  materials  tend  to  have  lower  variability  in  the  LWIR 
band  compared  to  the  MWIR  band; 

•  the  spectral  atmospheric  transmission  coefficient  does  not  fluctuate  significantly  in 
the  LWIR  band;  and 

•  the  optimal  temperature  in  the  LWIR  band  corresponds  to  a  value  near  ambient  air 
temperature  and  objects  in  simulations  tend  to  have  temperatures  near  ambient. 

The  validation  effort  has  shown  that  significant  errors  can  be  generated  by  the 
SensorVision  thermal  emission  model,  especially  for  the  MWIR  case.  However,  this 
statement  does  not  necessary  preclude  the  use  of  SensorVision  in  MWIR  simulations 
since  the  actual  level  of  errors  is  strongly  dependent  on  the  materials,  engagement 
scenarios,  and  actual  waveband  employed  in  the  simulation.  It  is  possible  to  minimise 
SensorVision  related  errors  by  carefully  constructing  simulations  using  the  El  as  a  guide. 
Appendix  A  contains  a  summary  explaining  the  use  of  El  values  during  simulation 
construction. 
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3.  Validation  of  the  Temperature  Prediction  Model^ 

In  the  previous  chapter,  the  SensorVision  thermal  emission  model  was  validated 
assuming  the  surface  temperatures  of  the  materials  were  known.  In  this  chapter,  the 
temperature  prediction  model  employed  by  SensorVision  will  be  experimentally 
validated  against  empirical  data. 

3.1  The  SensorVision  Temperature  Prediction  Model 

SensorVision  comprises  a  software  library  and  two  database  construction  tools,  called 
the  (MOSART  Atmospheric  Tool)  and  the  (Texture  Material  Mapper). 

The  software  library  performs  the  real-time  radiometric  calculations  and  provides  Vega 
with  the  required  functionality  to  render  IR  scenes.  Both  database  construction  tools  are 
used  in  preprocessing  operations  prior  to  the  real-time  simulation.  The  piupose  of  the 
TMM  utility  is  to  allow  the  user  to  assign  material  properties  (reflectance)  to  textures, 
which  are  pasted  on  object  models.  The  MAT  generates  databases  for  atmospheric 
quantities  and  surface  temperatures  of  scene  materials.  This  chapter  focxises  on  the  MAT 
since  it  is  the  SensorVision  component  responsible  for  computing  material  surface 
temperatures. 

The  MAT  is  essentially  a  SensorVision  interface  for  the  MOSART^  (Moderate  Spectral 
Atmospheric  Radiance  and  Transmittance)  code.  The  MAT  takes  a  user  input  file, 
executes  the  MOSART  program,  and  records  the  MOSART  output  results  into  database 
files  with  the  format  required  by  SensorVision.  Consequently,  the  validation  effort 
described  in  this  chapter  is  essentially  a  validation  of  the  temperature  prediction  model 
employed  by  the  MOSART  code  with  the  addition  of  the  MAT  interface.  The  component 
of  the  MOSART  program  that  performs  the  surface  temperature  computations  is  called 
the  TERTEM  (Terrain  Temperatures)  code.  The  TERTEM  algorithm  uses  a 
one-dimensional  three-layer  thermal  model  [4],  consisting  of  a  surface  layer  and  two 
sub-layers.  The  surface  material  has  some  prescribed  thickness  and  is  represented  by  six 
heat  transfer  parameters:  solar  absorption,  thermal  emittance,  thermal  conductance, 
specific  heat,  density,  and  characteristic  length.  Both  sub-layers  are  assigned  a  material 
type  with  the  first  sub-layer  given  a  thickness  property.  The  second  sub-layer  is  assumed 
to  be  infinite  in  depth.  The  TERTEM  algorithm  takes  into  accoimt  four  heat  transfer 
fluxes:  radiation,  convection,  conduction,  and  evaporation.  Radiation  inputs  are  from 
only  the  sun  and  the  sky.  Effects  of  varying  air  temperature  and  humidity  throughout 
the  day  are  considered,  but  the  wind  speed  and  moisture  content  of  materials  are 

Most  of  the  contents  in  this  chapter  have  already  been  reported  in  reference  [3].  It  has  been  included  in  this 
report  for  the  sake  of  completeness. 

MOSART  is  a  U.S.  Government  standard  atmospheric  code  developed  by  Phillips  Laboratory,  Department 
of  Geophysics,  Air  Force  Material  Command,  Handscom  AFB,  Massachusetts. 
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assumed  to  be  constant.  Also,  due  to  the  one-dimensional  model,  any  lateral  transfer  of 
heat  is  neglected.  The  TERTEM  program  should  yield  accurate  results  for  most  natural 
and  man-made  surfaces  (such  as  soils,  rocks,  low-cropped  vegetation,  asphalt,  and 
concrete).  However,  it  cannot  accurately  model  high  vertical  vegetation  (such  as  trees) 
due  to  the  dominant  effect  of  lateral  heat  transfer  processes,  or  operated  vehicles  (such 
as  tanks,  and  aeroplanes)  due  to  the  significant  irvfluences  of  internal  heat  sources  on 
surface  temperatures. 

The  SensorVision  predicted  temperature  results  used  in  this  validation  effort  where 
obtained  directly  from  the  MAT.  The  MAT  input  file  allows  the  user  to  specify  the 
atmospheric  states,  the  spectral  bands  of  interest  with  sensor  spectral  responses,  and  the 
range  of  material  surface  and  observer  geometries  expected  in  the  simulation.  Heat 
transfer  parameters  for  materials  can  also  be  modified  by  the  user.  Table  3-1  contains  the 
list  of  material  heat  transfer  parameters  that  can  be  adjusted  by  the  user,  while  Table  3-2 
contains  the  list  of  the  MAT  user  input  parameters.  Once  the  MAT  program  is  executed, 
three  output  files  are  generated:  a  file  containing  databases  of  solar  atmospheric 
quantities  (with  parameters  representing  solar  radiance,  skyshine  radiance,  atmospheric 
path  radiance,  and  atmospheric  path  transmission);  a  file  containing  databases  of  lunar 
atmospheric  quantities  (with  parameters  representing  lunar  radiance,  emitted  and 
skyshine  radiance,  emitted  and  scattered  atmospheric  path  radiance,  and  atmospheric 
path  transmission);  and  a  file  containing  the  database  of  predicted  surface  temperatures. 
Only  the  last  output  file  is  of  interest  in  this  report  since  it  contains  the  MAT  (and  the 
MOSART  code)  predicted  surface  temperatures  for  the  user-chosen  materials  given 
some  specified  geographical  location,  atmospheric  state,  and  spectral  band. 


Heat  Transfer  Parameter 


Label 

Default  Material 
Solar  Absorptivity 
Thermal  Emissivity 
Characteristic  Length 
Evaporation  Index 
Specific  Heat 
Thermal  Conductivity 
Density 

Surface  Layer  Thickness 
Sub-layer  1  Thickness 
Sub-layer  1  Material 
Sub-layer  2  Material 

Table  3-1:  List  of  heat  transfer  parameters. 
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In  this  chapter,  the  SensorVision  temperahire  prediction  models  are  validated  by 
comparing  the  MAT  generated  results  agamst  measured  temperatures.  This  type  of 
strategy  can  be  described  as  a  functional  and  experimental  validation.  The  term 
functional  refers  to  the  fact  that  the  physics  of  the  model  are  not  considered,  rather  only 
the  model  outputs  are  validated.  The  term  experimental  describes  the  fact  that  empirical 
data  is  used  to  validate  the  model.  The  measured  data  used  in  the  validation  were 
collected  in  trials  at  a  location  in  northern  Australia.  The  air  temperature  and  the 
temperature  of  a  concrete  sample  were  recorded  over  a  period  of  three  years  to  enable 
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the  monthly  averages  of  the  daily  temperature  profiles  to  be  determined.  SensorVision 
generated  temperature  profiles  were  then  compared  with  the  monthly  averages  to 
determine  the  accuracy  of  the  temperature  prediction  model.  All  simulations  were  rim 
using  version  3.1.1  of  the  MAT  (packaged  with  SensorVision  version  3.4.1.4,  and  Vega 
version  3.4),  on  an  C)nyx2™  SGI  machine  with  operating  system  IRIX™  6.5.  The  version 
of  the  MOSART  code  employed  by  SeiasorVision  is  not  known,  and  there  is  the 
possibility  that  the  MOSART  code  has  been  modified  by  the  SensorVision  developers. 

3.2  Processing  of  Experimental  Data 

The  measured  temperature  data  were  collected  in  1991,  1992,  and  1993,  during  an 
experimental  trial  with  a  purpose  unrelated  to  the  SensorVision  validation  effort.  During 
these  trials,  thermal  sensors  were  used  to  automatically  record  the  air  temperature  and 
the  temperature  of  a  concrete  sample  at  a  location  in  northern  Australia.  The  concrete 
specimen  was  a  350  mm  layer  of  Portland  cement  on  top  of  a  150  mm  layer  of  gravel.  Two 
temperature  sensors  were  buried  in  1  mch  holes  drilled  into  the  surface  of  the  concrete 
sample  at  different  spatial  locations.  The  holes  were  refilled  with  mortar.  One  air 
temperature  and  two  concrete  surface  temperature  readings  were  simultaneously 
recorded  every  10  minutes  throughout  each  24-hour  day.  Data  were  collected  for  every 
month  of  the  year,  except  December.  However,  not  every  month  of  each  year  or  every 
day  of  each  month  was  represented  in  the  sample  set.  Table  3-3  lists  the  days  in  which 
some  data  were  collected. 

Before  the  experimental  data  could  be  compared  against  SensorVision  generated  results, 
some  filtering  was  required  to  remove  any  data  sets  that  were  considered  to  be  unreliable 
or  erroneous.  A  temperatiue  reading  that  is  affected  by  a  thermal  sensor  malfunction  or 
is  an  anomalous  measurement  due  to  abnormal  conditions  is  referred  to  as  an  outlier. 
Whenever  an  outlier  was  identified  within  a  data  set,  the  entire  data  set  was  removed 
from  the  sample  space  in  order  to  reduce  the  complexity  of  the  statistics  calculations.  In 
addition,  incomplete  data  sets  containing  measurements  for  only  part  of  the  day  were 
also  removed  from  the  sample  space. 

Figure  3-1  (a)  shows  plots  of  the  raw  diurnal  air  and  concrete  temperature  profiles  for  the 
30  days  of  April  over  the  years  1991  to  1993.  As  evident  in  the  raw  data  plots  for  the 
concrete  sample,  several  data  sets  are  clearly  erroneous.  For  example,  several  data  sets 
contain  negative  temperature  readings,  which  are  improbable  given  the  tropical  cUmate 
of  northern  Australia.  In  comparison,  very  few  data  sets  in  the  air  temperature  profiles 
contain  obvious  outliers. 

Scrutiny  of  the  concrete  diurnal  temperature  profiles  revealed  that  there  were  two  types 
of  obvious  outliers  associated  with  sensor  malfunctions.  A  temperature  of  99.9  °C  was 
recorded  when  a  failure  in  the  thermal  sensor  occurred,  while  a  large  temperature 
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variation  between  consecutive  temperature  readings  indicated  a  DC  (direct  current)  bias 
problem  within  the  sensor.  Both  outlier  types  were  easily  identified  and  the  data  sets 
containing  these  outliers  were  removed  from  the  sample  space.  The  identification  of 
other  outlier  types  was  difficult  due  to  an  incomplete  knowledge  of  the  sensor 
characteristics,  the  measurement  process,  and  the  conditions  imder  which  each 
temperature  reading  was  recorded.  It  was  not  easy  to  determine  if  a  peculiar  feature  of  a 
temperature  profile  was  due  to  an  erroneous  measurement  or  due  to  a  valid 
environmental  condition.  The  only  information  that  could  be  used  in  the  detection  of 
non-obvious  outliers  was  the  fact  that  three  temperature  readings  (one  air  temperature 
and  two  concrete  temperatures)  were  recorded  simultaneously.  AH  three  diurnal 


1991 

1992 

1993 

January 

30,31 
(2  days) 

February 

1,4,5, 18, 19 

12-28 

(5  days) 

(17  days) 

March 

26-31 

2-31 

1-31 

(6  days) 

(30  days) 

(31  days) 

April 

1-30 

1-30 

1-30 

(30  days) 

(30  days) 

(30  days) 

May 

1-31 

1-31 

1-31 

(31  days) 

(31  days) 

(31  days) 

June 

1-30 

1-30 

1-30 

(30  days) 

(30  days) 

(30  days) 

July 

1-31 

1-31 

1-31 

(31  days) 

(31  days) 

(31  days) 

August 

1-31 

1-31 

1-31 

(31  days) 

(31  days) 

(31  days) 

September 

1-30 

1-30 

1-26 

(30  days) 

(30  days) 

(26  days) 

October 

1-31 

1-26, 28-31 

(31  days) 

(30  days) 

November 

1-11 

1-9, 12-30 

(11  days) 

(28  days) 

December 

Table  3-3;  List  of  days  during  1991, 1992,  and  1993  in  which  some  temperature 
measurements  were  recorded. 
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temperature  profiles,  associated  with  a  particular  day  and  data  set,  should  be  reasonably 
well  correlated  because  they  were  all  measured  under  identical  environmental 
conditions.  In  particular,  the  two  concrete  temperature  profiles  recorded  during  any  day 
shoTild  have  a  high  degree  of  correlation.  An  outlier  can  therefore  be  recognised  as  a 
temperature  profile  feature  that  is  not  mimicked  in  the  other  two  temperature  profiles 
measured  during  that  same  day.  The  three  diurnal  temperature  profiles  were  plotted  and 
compared  for  aU  data  sets,  and  the  identification  of  outliers  (by  a  human  operator)  was 


Figure  3-1:  Plots  of  the  (a)  raw,  (b)  filtered,  and  (c)  the  statistics  of  the  data  sample 
space  for  measured  air  and  concrete  daily  temperature  profiles  for  the  month  of  April 
(1991, 1992,  and  1993).  The  data  statistics  areplotted  as  error  bars  with  a  mean  and 
an  error  of  one  standard  deviation. 
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performed.  The  process  was  subjective  and  conducted  with  the  policy  that  only  data  sets 
with  features  that  were  blatant  outliers  were  discarded.  Figure  3-2  contains  examples  of 
diurnal  temperature  variations  for  two  days.  Figure  3-2(a)  shows  temperature  profiles  in 
which  no  outliers  were  identified,  and  Figure  3-2(b)  shows  an  example  of  an  outlier 
feature.  In  the  first  case,  aU  three  plots  are  well  correlated  while  in  the  second  case,  the 
dashed  line  contains  a  spike  near  4  p.m.  that  is  not  evident  in  the  other  two  temperature 
profiles.  The  data  set  corresponding  to  the  dashed  line  in  Figure  3-2(b)  was  removed 
from  the  sample  space.  Note  that  the  sudden  decrease  in  temperature  observed  near  3 
p.m.  in  Figure  3-2(b)  may  have  been  misinterpreted  as  an  outlier  if  each  temperature 
profile  was  individually  scrutinized.  However,  because  aU  three  temperature  profiles 
measxured  that  day  exhibited  the  same  feature,  it  must  be  concluded  that  some  valid 
environmental  effect  (such  as  a  cool  change  or  sudden  rainfall)  caused  the  rapid  decrease 
in  temperature. 


Table  3-4  lists  the  total  number  of  raw  data  sets,  the  number  of  data  sets  rejected  on  the 
basis  that  the  records  were  incomplete  or  contained  outliers,  and  the  final  sample  space 
size  for  each  month  (combining  the  1991,  1992,  and  1993  data).  Note  that  the  large 
numbers  of  rejected  data  sets  in  the  concrete  sample  space  is  mainly  due  to  temperature 
readings  that  indicated  sensor  malfunctions.  The  embedding  of  the  thermal  sensors  into 
the  concrete  sample  caxised  a  reduction  in  the  reliability  of  the  sensors.  With  reference  to 
Table  3-4,  the  final  sample  space  size  for  January  is  too  small  to  be  useable,  hence  January 
was  not  considered  during  the  validation  procedmre.  The  number  of  data  sets  for 


(a) 


(b) 


Figure  3-2;  Examples  of  the  temperature  profile  comparisons  used  to  identify  outliers: 
(a)  temperature  profiles  for  28  April  1992,  and  (b)  temperature  profiles  for  the 
5  February  1992,  showing  an  outlier  in  the  sensor  #2  concrete  temperature  profile. 
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February,  March  and  November  were  considered  adequate  while  the  size  of  the  sample 
spaces  associated  with  the  remaining  months  (excluding  December)  were  deemed  more 
than  sufficient  for  the  validation  purpose.  It  is  assumed  that  the  sample  space  for  every 
month  except  January  and  December  is  sufficiently  large  such  that  any  atypical  or 
imusual  feature  of  a  diurnal  temperature  curve  will  be  averaged  out  in  the  statistics,  so 
as  to  not  significantly  affect  the  mean  results  for  a  particular  month. 

Figure  3-1  (b)  contains  plots  of  the  filtered  data  for  the  air  and  concrete  daily  temperature 
profiles  for  the  month  of  April.  In  particular,  note  that  the  filtered  data  for  the  concrete 
temperature  profiles  are  significantly  cleaner  than  the  original  raw  data  (shown  in 
Figure  3-l(a)).  The  filtered  data  for  the  air  temperature  case  does  not  vary  significantly 
from  the  raw  data  because  the  thermal  sensor  recording  the  air  temperatures  was  not  as 
prone  to  malfunctions  as  the  thermal  sensors  measuring  the  concrete  temperatures. 

Having  filtered  the  experimental  data  for  outliers,  the  mean  and  standard  deviation  of 
the  temperature  profiles  were  computed  for  each  month.  The  statistics  of  the  air  and  the 
concrete  temperature  profiles  for  the  month  of  April  are  shown  in  Figure  3-1  (c).  The  error 


-  Air  Temperature  Profiles 

Concrete  Temperature  Profiles 

Data 

Sets 

Rejected 

Sets 

Total 

Data 

Sets 

Rejected 

Sets 

Total 

January 

2 

1 

1 

4 

2 

2 

February 

22 

4 

18 

44 

12 

32 

March 

67 

4 

63 

134 

54 

80 

April 

90 

3 

87 

180 

41 

139 

May 

93 

3 

90 

186 

40 

146 

June 

90 

4 

86 

180 

38 

142 

July 

93 

2 

91 

186 

35 

151 

August 

93 

1 

92 

186 

33 

153 

September 

86 

4 

82 

172 

34 

138 

October 

61 

3 

59 

122 

9 

113 

November 

39 

3 

36 

78 

27 

51 

December 

- 

- 

- 

- 

- 

- 

Table  3-4:  Summary  of  sample  space  sizes  for  each  month  listing  the  number  of  raw  data  sets, 
the  number  of  data  sets  that  were  incomplete  or  contained  outliers,  and  the  final  total  sample 

space  size. 
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bars  in  these  plots  represent  the  mean  profile  with  an  error  of  one  standard  deviation. 
Assuming  an  unbiassed  sample  space  and  Gaussian  statistics,  68.25  %  of  all 
temperatures  (measured  or  predicted  for  this  particular  location)  would  be  expected  to 
lie  within  one  standard  deviation  of  the  mean  at  any  instant  in  time.  A  proportion  of 
95.46  %  and  99.73  %  should  lie  within  two  and  three  standard  deviations  of  the  mean, 
respectively.  The  plots  of  the  statistics  for  the  months  from  February  to  November  can  be 
foimd  in  Figure  3-5  and  Figure  3-6. 

3.3  Validation  of  Temperature  Prediction  Model 

Complete  24-hour  diurnal  temperature  profiles  were  computed  using  the  MAT  for  the 
materials  air  and  concrete  for  the  specified  longitudinal  and  latitudinal  coordinates  of 
the  location  in  northern  Australia  being  considered.  Air  temperatxires  were  computed  by 
stipulating  a  blackbody  in  the  material  specification  with  an  altitude  of  108  m.  This 
altitude  was  chosen  because  the  location  in  northern  Australia  has  an  elevation  of  108  m 
above  sea  level.  Results  were  obtained  for  every  month  of  the  year,  and  the  standard  heat 
transfer  parameter  files  included  with  the  SensorVision  package  were  employed  in  the 
simulations. 

Several  MAT  characteristics  were  noted  during  the  validation  process.  In  the  case  of  the 
air  temperahue  computations,  the  same  MAT  results  were  obtained  regardless  of  the 
year  specified.  In  addition,  the  temperatures  were  identical  for  the  first  day  to  the 
penultimate  day  of  each  month.  Temperature  predictions  for  the  last  day  of  each  month 
matched  the  results  for  the  next  month,  provided  the  specified  month  was  February, 
April,  June,  Jtdy,  or  November.  Temperature  predictions  for  the  last  day  of  January, 
March,  May,  August,  September  and  October  were  unique  within  the  year.  The  MAT 
results  for  all  days  in  December  were  identical.  Figure  3-3  summarises  the  MAT 
characteristics  described  above  by  graphically  illustrating  the  days  of  the  year  for  which 
identical  air  temperahue  profiles  are  predicted.  In  the  case  of  the  concrete  temperahue 
profiles,  results  were  unique  for  aU  days  in  the  year.  However,  the  variations  from  one 
day  to  the  next  were  minor  and  incremental  except  between  days  that  also  experienced 
a  change  in  the  air  temperahue  predictions.  Differences  in  results  were  noted  between 
years  but  these  differences  were  considered  negligible.  The  reasons  for  these  peculiar 
patterns  of  predicted  temperahue  profiles  as  a  function  of  the  day  of  year  are  tmknown. 
Although  these  MAT  characteristics  have  no  perceivable  affect  on  the  validation  effort, 
they  were  mentioned  above  as  a  matter  of  curiosity  and  for  the  sake  of  completeness.  In 
the  validation  effort,  the  air  and  concrete  siuface  temperatiues  were  computed  for  every 
horu  diuing  the  15th  day  of  each  month  for  the  year  1991.  These  results  were  considered 
representative  of  typical  MAT  predicted  values  for  each  month  of  the  year.  Other 
pertinent  MAT  input  parameters  used  in  the  simulations  include:  the  default 
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atmospheric  model  for  the  location  and  day  of  year;  mean  temperature,  humidity,  and 
wind;  clear  cloud  cover  with  a  cloud  base  altitude  of  1  km;  and  a  visibility  range  of  23  km. 

Initially,  the  MAT  predicted  temperatures  were  compared  against  records  obtained  from 
the  (Australian)  Bureau  of  Meteorology.  For  the  particular  location  in  northern  Australia, 
averaged  9  a.m.  and  3  p.m.  air  temperatures  for  each  month  (collected  over  14  and  13 
years,  respectively)  were  available.  Figure  3-4  shows  the  comparisons  between  the  MAT 
predicted  air  temperatures  and  the  Bureau  of  Meteorology  data.  As  evident  in  the  plots, 
reasonable  agreement  is  attained  for  the  9  a.m.  air  temperature  results  for  all  months, 
except  September,  October  and  November.  The  MAT  consistently  imder-predicts  the 
3  p.m.  air  temperatures  for  this  location. 


The  MAT  generated  diurnal  temperature  profiles  and  the  experimental  data  (processed 
in  Section  3.2)  are  plotted  in  Figure  3-5  and  Figure  3-6.  The  air  temperature  diurnal 
profiles  are  shown  in  Figure  3-5  while  Figure  3-6  contains  the  concrete  diurnal 
temperature  profiles.  It  is  apparent  from  these  plots  that  the  MAT  generated  results  do 
not  compare  weU  to  the  experimental  data.  With  regard  to  the  air  temperature  profiles, 
the  predicted  results  have  significantly  smaller  diurnal  ranges  than  that  exhibited  by  the 
meastued  data.  There  is  a  provision  in  the  MAT  program  to  include  an  extension  in  the 
MAT  input  file  to  calibrate  the  computed  temperature  profiles.  In  this  extension,  an  air 
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Figure  3-3:  Diagram  showing  the  days  of  the  year  for  which  the  MAT  predicts 
identical  daily  temperature  profiles. 
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temperahire  and  time  of  day  are  specified.  The  air  temperature  curves  are  then 
calibrated,  such  that  the  specified  air  temperature  is  obtained  at  the  specified  time.  The 
net  effect  of  the  extension  is  to  move  the  predicted  temperature  profiles  up  or  down,  but 
there  is  negligible  effect  on  the  diurnal  range.  Consequently,  the  MAT  generated 
temperatures  cannot  be  made  to  coincide  with  the  empirical  data  to  any  acceptable  level. 
This  problem  is  attributed  to  differences  between  the  standard  atmospheric  (tropical 
summer  and  tropical  winter)  models  employed  by  the  MOSART  code  and  the  actual 
atmospheric  profile  at  the  location  xmder  consideration.  The  MOSART  code  was 
predominantly  developed  for  application  to  northern  hemisphere  regions,  hence 
problems  with  its  application  to  this  southern  hemisphere  tropical  zone  was  not  entirely 
xmexpected.  Discussions  with  MultiGen-Paradigm  have  revealed  that  a  modification  in 
the  MOSART  code  is  planned  in  future  upgrades  of  the  MAT,  which  will  allow  iisers  to 
define  appropriate  atmospheric  profiles  for  a  geo-specific  location.  This  functionality  is 
expected  to  appear  in  version  3.5  of  SensorVision.  Until  user-specified  atmospheric 
profiles  are  supported,  it  is  concluded  that  the  MAT  cannot  satisfactorily  predict  air 
temperatures  for  the  geo-specific  location  in  northern  Australia  being  considered. 

The  predicted  concrete  temperature  profiles  also  show  poor  agreement  with  the 
experimental  data.  Although  the  shape  of  the  curves  are  similar,  most  of  the  predicted 
temperature  data  lie  outside  of  the  error  bars  plotted  in  Figiire  3-6.  However,  unlike  the 
air  temperature  case,  the  error  in  the  MAT  predicted  concrete  temperatures  may  be  due 
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Figure  S-4:  Comparisons  of  the  MAT  generated  results  with  mean  9  a.m.  and  3  p.m.  air 
temperatures  obtained  from  the  (Australian)  Bureau  of  Meteorology . 
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Figure  3-5:  Comparison  between  the  MAT  generated  and  measured  diurnal  air  temperature 
profiles  for  the  months  from  February  to  November. 
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Figure  3-6:  Comparison  between  the  MAT  generated  and  measured  diurnal  concrete 
temperature  profiles  for  the  months  from  February  to  November. 
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to  incorrect  heat  transfer  parameters  for  the  actual  concrete  sample  used  in  the  trials.  In 
a  more  rigorous  validation  effort,  the  heat  transfer  parameters  for  the  concrete  sample 
would  be  measured.  Unfortunately,  physical  access  to  the  concrete  sample  was  not 
possible,  hence  a  more  heuristic  approach  of  approximating  the  heat  transfer  parameters 
was  adopted  [5].  The  heat  trarrsfer  parameters  were  modified  until  a  good  match  was 
achieved  between  the  predicted  concrete  temperature  profiles  and  the  empirical  data. 
The  effects  of  individually  varying  solar  absorptivity,  thermal  emissivity,  specific  heat, 
thermal  conductivity,  and  material  density  on  the  concrete  diurnal  temperature  profile 
computed  for  April  are  shown  m  the  plots  of  Figure  3-7.  Nearly  aU  features  of  the 
temperature  profile  could  be  varied.  The  exception  was  the  temperature  minimum 
(occurring  around  7  a.m.),  which  could  not  be  made  to  coincide  well  with  the  empirical 
data  while  simultaneously  maintaining  a  realistic  thermal  emissivity  value.  This 


Temperature  band  bound  by  one 

. .  standard  deviation  above  and  below 

the  mean  measured  temperature 

The  two  numbers  displayed  on  the  right  side  of 
each  family  of  curves  indicate  the  minimum  and 
maximum  heat  transfer  parameter  values  used  to 
generate  the  plots.  The  steps  between  the 
minimum  and  maximum  values  are  constant. 


Figure  3-7:  Graphs  illustrating  the  effect  of  varying  heat  transfer  parameters  on  the  MAT 
predicted  concrete  diurnal  temperature  profiles  for  the  month  of  April. 
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problem  is  most  likely  due  to  the  MAT  incorrectly  predicting  the  air  temperatures.  A  fix 
was  obtained  by  exploiting  the  MAT  extension  (described  above)  to  artificially  move  the 
temperature  profiles  up.  The  use  of  the  MAT  extension  is  only  a  superficial  solution  in 
the  sense  that  it  would  be  ineffective  if  more  than  one  material  was  specified  in  the  MAT 
input  file.  A  proper  solution  to  the  problem  requires  modifications  to  be  made  to  the 
MOSART  program  to  ensure  that  air  temperatures  are  accurately  predicted  for  the 
geo-specific  location  considered. 

Using  essentially  a  trial-and-error  process,  a  combination  of  heat  transfer  parameters 
that  resulted  in  a  good  general  agreement  between  the  MAT  generated  concrete 
temperature  profiles  and  the  measured  data  was  obtained.  The  modified  heat  transfer 
parameter  values  are  listed  in  Table  3-5.  Note  that  these  values  are  sensible  and  realistic. 
The  significant  change  in  value  of  thermal  conductivity  of  concrete  from  1.73  to  0.5 
Wm'^K'^  is  reasonable,  given  that  a  figure  of  0.84  Wm'^K'^  was  quoted  in  a  textbook  [6]. 
The  final  concrete  temperature  profiles  are  plotted  in  Figure  3-8.  Note  that  the  values  of 
the  7  a.m.  temperatures  defined  in  the  MAT  extensions  were  varied  for  each  month  to 
cotmter  the  problem  of  incorrectly  predicted  air  temperatures.  Furthermore,  the  values 
employed  were  imrealistic  (ranging  from  27  °C  to  42  °C)  due  to  the  fact  that  the  MAT 
generated  air  temperatures  had  an  erroneous  diurnal  range  compared  with  the  empirical 
data.  The  7  a.m.  value  used  in  the  MAT  extension  for  each  month  is  indicated  in  the  top 


Heat  Transfer  Parameter 

Original 

Modified 

Label 

Concrete 

Concrete 

Default  Material 

Concrete 

Concrete 

Solar  Absorptivity 

0.6681411 

0.4 

Thermal  Emissivity 

0.9333010 

0.9 

Characteristic  Length 

1.00  m 

1.00  m 

Evaporation  Index 

No 

No 

Specific  Heat 

0.88  J  gm'l  K-’ 

0.7  J  gm'*  K'^ 

Thermal  Conductivity 

1.73  Wm-l  K-’ 

0.5  W  m-*  K'^ 

Density 

2.4  X  10"^  gm  m'^ 

2.3  X  10'^  gm  m‘^ 

Surface  Layer  Thickness 

0.35  m 

0.35 

Sub-layer  1  Thickness 

0.15  m 

0.15 

Sub-layer  1  Material 

Road  Gravel 

Road  Gravel 

Sub-layer  2  Material 

Packed  Dirt 

Packed  Dirt 

Table  3-5:  List  of  original  and  modified  heat  transfer  parameters. 
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Figure  3-8:  Comparison  between  the  MAT  results  with  modified  concrete  heat  transfer 
parameters  and  measured  diurnal  concrete  temperature  profiles. 
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right  comer  of  the  relevant  plots  in  Figure  3-8.  The  overall  agreement  of  the  MAT 
predicted  concrete  temperatures  using  the  modified  heat  transfer  parameters  with  the 
measured  data  is  extremely  good  for  most  months.  The  poorest  agreement  was  obtained 
in  Febmary,  and  these  results  are  still  considered  reasonable.  Recall  that  the  size  of  the 
sample  space  of  the  measured  data  in  Febmary  was  only  considered  adequate,  hence  a 
high  level  of  confidence  is  not  associated  with  the  statistics  corresponding  to  this  month. 

The  variation  of  material  heat  transfer  parameters  to  obtain  an  acceptable  match  between 
measured  and  predicted  temperature  data  is  a  feasible  technique  for  generating 
validated  material  temperature  profiles.  However,  the  method  can  only  be  employed 
once  the  problem  with  the  air  temperature  predictions  has  been  fixed. 

3.4  Chapter  Summary 

Results  of  an  experimental  validation  of  the  SensorVision  temperature  prediction  model 
were  presented  in  this  chapter.  SensorVision  generated  diurnal  air  and  concrete 
temperature  profiles  were  compared  against  experimental  data  obtained  from  a  location 
in  northern  Australia.  It  was  observed  that  the  SensorVision  results  did  not  compare  well 
with  the  empirical  data.  In  particular,  the  predicted  diurnal  air  temperature  ranges  were 
significantly  smaller  than  those  measured.  These  inaccuracies  were  attributed  to 
differences  between  the  actual  atmospheric  profile  at  the  geo-specific  location  and  the 
standard  atmospheric  models  employed  by  the  MOSART  code.  This  problem  should  be 
resolved  in  future  software  upgrades  of  SensorVision,  which  are  expected  to  include 
functionality  for  users  to  custom  define  atmospheric  profiles.  Until  user-defined 
atmospheric  profiles  are  supported  in  the  MAT,  the  physical  accuracy  of  the  temperature 
models  employed  by  SensorVision  cannot  be  properly  evaluated. 

Predicted  concrete  temperature  profiles  compared  well  with  the  empirical  data  after 
modifications  were  made  to  the  concrete  heat  transfer  parameters.  However,  it  was 
necessary  to  use  a  MAT  extension  to  artificially  inflate  air  temperatures  in  order  to 
achieve  good  agreement  between  predicted  and  measured  results.  The  use  of  the  MAT 
extension  is  only  a  superficial  solution  in  the  sense  that  the  air  temperature  values 
employed  were  unrealistic,  and  the  use  of  the  extension  would  be  ineffective  if  more  than 
one  material  was  specified  m  the  MAT  input  file.  A  proper  solution  to  the  problem 
requires  modifications  to  be  made  to  the  MOSART  program  to  ensure  that  air 
temperatures  are  accurately  predicted  for  the  geo-specific  location  considered. 
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4.  Conclusions  and  Further  Work 

This  report  presented  the  results  from  a  validation  conducted  on  the  SensorVision 
thermal  emission  model.  The  thermal  emission  model  essential  involves  two 
components:  the  prediction  of  an  object's  surface  temperature  given  the  surrotmding 
conditions;  and  the  calculation  of  the  radiance  incident  on  a  sensor  due  to  that  object  as 
a  result  of  thermal  emission  processes.  The  validation  effort  described  in  this  report 
considered  the  temperahire  prediction  model  as  weU  as  the  radiance  computation 
model. 

The  SensorVision  radiance  computation  model  was  validated  mathematically  against  an 
accurate  non-real  time  thermal  emission  model.  SensorVision  employs  an 
approximation  whereby  spectral  terms  within  the  thermal  emission  equation  are  moved 
outside  of  the  spectral  integration  and  replaced  by  their  in-band  averaged  values.  This 
approximation  is  required  to  enable  real-time  operation.  The  effect  of  the  approximation 
on  the  accuracy  of  the  SensorVision  thermal  emission  model  was  analysed  and  a  figure 
of  merit  was  developed  to  enable  quick  determination  of  likely  error  levels.  The  figure  of 
merit  was  based  on  the  second  moment  of  the  spectral  variables  and  was  referred  to  as 
the  error  indicator  or  El.  The  main  purpose  of  the  El  is  to  aid  users  during  simulation 
construction  to  identify  scenario  components  that  may  impact  on  the  integrity  of  the  HIL 
simulation.  The  main  conclusion  that  can  be  drawn  from  the  validation  of  the  radiance 
computation  model  is  that  SensorVision  appears  to  be  more  suited  to  LWIR  simulations 
than  MWIR  simulations.  The  reasons  for  this  conclusion  are  three-fold: 

•  the  variability  of  the  spectral  emissivities  of  materials  tend  to  be  lower  in  the  LWIR 
band  compared  to  the  MWIR  band; 

•  the  spectral  atmospheric  transmission  coefficient  does  not  vary  much  in  the  LWIR 
band  compared  to  the  MWIR  band;  and 

•  simulations  tend  to  involve  objects  with  temperatures  near  ambient  temperature, 
which  results  in  smaller  error  levels  in  the  LWIR  band  and  higher  error  levels  in 
the  MWIR  band. 

Given  that  IR  phenomenology  in  the  LWIR  band  is  emission  dominated,  it  can  be 
concluded  that  SensorVision  can  be  coiifidently  employed  in  HIL  simulations  involving 
LWIR  seekers  (provided  the  prediction  or  assignment  of  surface  temperatures  is 
accurate).  HIL  simulatioi^s  involving  MWIR  seekers,  using  SensorVision,  should  be 
constructed  with  care.  The  validation  has  shown  that  significant  errors  can  be  generated 
by  the  SensorVision  thermal  emission  model  in  the  MWIR  band.  These  errors  can  be 
mitigated  by  carefully  choosing  the  materials  and  engagement  geometries  employed  in 
the  simulation  scenarios.  Furthermore,  simulations  associated  with  narrower  bands  than 
the  full  3  to  5  pm  band  tested  during  the  validation  are  likely  to  yield  lower  levels  of 
error.  Confidence  in  the  use  of  SensorVision  for  MWIR  HIL  simulations  requires  the 


47 


DSTO-RR-0212 


validation  of  reflection  components  of  the  IR  radiometric  equation,  since  reflection 
phenomena  are  significant  in  the  MWIR  band. 

SensorVision  employs  a  standard  atmospheric  program  called  the  MOSART  code  to 
predict  sturface  temperatures  of  objects  prior  to  the  real-time  simulation.  The 
temperature  prediction  model  was  validated  agamst  experimental  data  obtained  from 
trials  conducted  m  northern  Australia.  During  these  trials,  daily  temperature  profiles 
were  measured  for  air  and  a  concrete  sample.  In  the  validation,  the  MOSART  predicted 
temperatures  were  compared  against  empirical  data  and  it  was  shown  that  the 
temperature  prediction  model  was  not  accmate  for  the  particular  location  in  northern 
Australian.  The  reason  for  the  poor  agreement  between  the  predicted  and  empirical  data 
was  attributed  to  differences  between  the  tropical  atmospheric  model  used  by  the 
MOSART  code  and  the  atmospheric  profile  at  the  location  in  northern  Australia.  The 
inability  of  SensorVision  to  allow  users  to  specify  custom  atmospheric  profiles  was 
considered  a  significant  problem  in  the  ctirrent  software.  Fortunately,  the  next  release  of 
SensorVision  is  expected  to  provide  the  functionality  for  user  defined  atmospheric 
profiles.  The  results  for  the  concrete  temperature  profile  comparisons  showed  the 
importance  of  providing  correct  material  heat  transfer  parameters.  Heat  transfer 
parameters  can  be  derived  heuristically  by  matching  the  SensorVision  generated 
temperature  profiles  with  the  empirical  data. 

Further  work  is  required  to  complete  the  verification  and  validation  of  SensorVision.  In 
particular,  the  following  tasks  still  need  to  be  performed. 

•  The  reflection  components  of  the  IR  radiometric  equation  need  to  be  validated  to 
determine  its  effect  on  MWIR  simulations. 

•  Verification  of  the  SensorVision  algorithm  is  still  required.  The  accuracy  associated 
with  the  use  of  lookup  tables  and  the  exactitude  of  employing  extrapolation 
techniques  to  approximate  various  parameters  should  be  investigated. 

•  The  effects  of  digital  artifacts  on  HIL  simulations  should  be  explored  [1].  The 
effects  of  alias  induced  scintillations,  pixel  bit  resolution,  and  z-buffer  fighting 
have  the  potential  to  compromise  the  integrity  of  HIL  simulation  results. 

•  Finally,  procedures  for  the  validation  of  particular  SensorVision  simulations 
should  be  proposed.  In  particular,  accurate  and  realistic  non-real-time  models 
should  be  developed  or  acquired  for  the  purposes  of  cross-validation  in 
post-processing  HIL  simulation  activities. 
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Appendix  A:  Using  Error  Indicators 

The  purpose  of  error  indicators  is  to  help  users  determine  the  likely  level  of  errors 
associated  with  using  SensorVision  in  tike  generation  of  IR  simulations.  All  error 
indicators  have  a  possible  range  of  zero  to  100,  where  a  value  of  100  represents  the  worst 
case.  There  are  error  indicators  associated  with  the  spectral  emissivity  of  materials,  the 
spectral  atmospheric  transmission  coefficient,  and  the  Planck's  blackbody  spectral 
radiance  parameter.  Furthermore,  the  error  indicators  can  be  combined  to  give  the 
collective  effect  of  spectral  variations  in  the  material  emissivity,  the  atmospheric 
transmission  coefficient,  and  the  Planck's  blackbody  curve  on  the  SensorVision 
generated  errors.  This  appendix  provides  the  equations  and  data  required  to  use  error 
indicators  for  a  range  of  possible  situations.  The  wavebands  considered  include  the 
MWIR  bemd  (nominally  specified  as  the  band  from  3  to  5  pm)  and  the  LWIR  band 
(nominally  specified  as  the  band  from  8  to  12  pm). 

A.l  Error  Indicators  for  Material  Emissivities 

The  El  values  for  the  spectral  emissivities  of  materials  can  be  calculated  using 

£4  =  200e^,  (A-1) 

where  is  the  second  moment  of  the  spectral  emissivity  data.  The  second  moments  for 
a  range  of  materials  are  listed  in  Table  A-1,  Table  A-2,  Table  A-3,  Table  A-4,  and 
Table  A-5. 


Composite  Materials 

3  -  5  pm  band 

8-12  pm  band 

£ 

£ 

beige_fabric 

0.3789 

0.0297 

0.2705 

0.0249 

black_rubber 

0.0428 

0.0013 

0.0468 

0.0036 

brown_plastic 

0.1085 

0.0419 

0.0526 

0.0101 

cream_fiberglass 

0.0468 

0.0010 

0.0467 

0.0060 

gold_nylon 

0.1647 

0.0863 

0.0861 

0.0327 

green_canvas 

0.0675 

0.0255 

0.0379 

0.0133 

grey_fabric 

0.3926 

0.0276 

0.3513 

0.0470 

olive_plastic 

0.1378 

0.0453 

0.0251 

0.0014 

orange_epoxy 

0.0363 

0.0010 

0.0385 

0.0023 

red_nylon 

0.0460 

0.0055 

0.0440 

0.0040 

tan_felt 

0.1743 

0.0945 

0.0580 

0.0242 

white_fabric 

0.2279 

0.0446 

0.0241 

0.0035 

yellow_silicon 

0.1578 

0.0761 

0.0751 

0.0337 

asphalt_shingles 

0.0684 

0.0198 

0.0540 

0.0210 

Table  A-1:  Emissivity  mean  and  second  moment  values  for  different  composite  materials. 
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Construction 

Materials 

3-5  pm  band 

8-12  pm  band 

£ 

E 

black_asphalt 

0.0516 

0.0097 

0.0531 

0.0136 

block_concrete 

0.1064 

0.0306 

0.0573 

0.0169 

clear_glass 

0.0305 

0.0080 

0.1888 

0.1231 

dark_titanium 

0.6908 

0.0197 

0.7889 

0.0098 

metal_roof 

0.7930 

0.0298 

0.8731 

0.0068 

oxydized_aluminum 

0.7631 

0.0674 

0.8065 

0.0193 

paved_concrete 

0.0561 

0.0183 

0.0334 

0.0107 

pebbled_asphalt 

0.0684 

0.0198 

0.0540 

0.0210 

pine_wood 

0.1220 

0.0509 

0.0537 

0.0181 

red_brick 

0.2739 

0.0870 

0.0424 

0.0226 

roof_tar 

0.0488 

0.0016 

0.0418 

0.0016 

rusting_steel 

0.6963 

0.0330 

0.8206 

0.0120 

urban_commercial 

0.0491 

0.0170 

0.0273 

0.0041 

urban_residentia! 

0.1209 

0.0582 

0.0689 

0.0038 

wood_siding 

0.1012 

0.0344 

0.0438 

0.0083 

Table  A-2:  Emissivity  mean  and  second  moment  values  for  different  construction  materials. 


Paints 

3-5  pm  band 

8-12  pm  band 

E 

£ 

ash  _grey 

0.0667 

0.0064 

0.0483 

0.0093 

battleship _grey 

0.0241 

0.0012 

0.0346 

0.0082 

blackish_brown 

0.1589 

0.0405 

0.0636 

0.0181 

bleached_yellow 

0.0792 

0.0199 

0.0231 

0.0049 

canary  _yellow 

0.0814 

0.0214 

0.0541 

0.0098 

dark_tan 

0.1088 

0.0329 

0.0560 

0.0162 

jet_black 

0.0412 

0.0013 

0.0531 

0.0064 

kelly_green 

0.0457 

0.0022 

0.0471 

0.0070 

light_red 

0.0708 

0.0223 

0.0291 

0.0086 

light_tan 

0.0852 

0.0101 

0.0548 

0.0135 

mint_green 

0.0450 

0.0114 

0.0208 

0.0066 

mustard_yellow 

0.3008 

0.0871 

0.1271 

0.0164 

off_white 

0.1076 

0.0484 

0.0447 

0.0092 

olive_green 

0.0926 

0.0223 

0.0537 

0.0122 

sky_blue 

0.0918 

0.0211 

0.0321 

0.0091 

snow_white 

0.2060 

0.0992 

0.0540 

0.0183 

Table  A-3:  Emissivity  mean  and  second  moment  values  for  different  types  of  paints. 
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Soils 

3  -  5  fim  band 

8-12  (tm  band 

8 

8 

beach_sand 

0.3075 

0.1456 

0.0650 

0.0334 

black_sand 

0.0886 

0.0176 

0.0779 

0.0445 

clay_soil 

0.2835 

0.0808 

0.0761 

0.0402 

compact_soil 

0.1034 

0.0319 

0.0409 

0.0070 

desert_sand 

0.1677 

0.0692 

0.0610 

0.0215 

dry_lakebed 

0.3115 

0.0929 

0.0650 

0.0334 

dry_seabed 

0.3609 

0.1058 

0.0634 

0.0318 

fresh_snow 

0.0170 

0.0052 

0.0080 

0.0038 

ice 

0.0225 

0.0121 

0.0163 

0.0089 

lake_sand 

0.1379 

0.0649 

0.0916 

0.0470 

limestone-silt 

0.1631 

0.0245 

0.0834 

0.0248 

limestone-silt-sand 

0.1269 

0.0125 

0.0515 

0.0084 

limestone-silt-soil 

0.1808 

0.0388 

0.0756 

0.0147 

limestone_rock 

0.1459 

0.0293 

0.0924 

0.0367 

loam_soil 

0.1110 

0.0425 

0.0286 

0.0049 

loamy_sand 

0.1763 

0.0866 

0.1221 

0.0846 

lump_coal 

0.0269 

0.0028 

0.0212 

0.0026 

old_snow 

0.0152 

0.0050 

0.0093 

0.0034 

road _gravel 

0.1191 

0.0511 

0.0496 

0.0157 

salt-silt 

0.1720 

0.0352 

0.1016 

0.0022 

sand-soil 

0.1649 

0.0379 

0.0483 

0.0134 

sandstone-soil 

0.1163 

0.0226 

0.0565 

0.0152 

sandstone_rock 

0.1459 

0.0293 

0.0924 

0.0367 

sandyjoam 

0.1793 

0.0635 

0.0557 

0.0303 

scrub-soil 

0.1808 

0.0388 

0.0756 

0.0147 

scrub-soil  (shaded) 

0.1507 

0.0562 

0.0216 

0.0024 

silt-sand 

0.2700 

0.0863 

0.0433 

0.0159 

silty_clay 

0.0725 

0.0224 

0.0286 

0.0047 

siltyjoam 

0.3609 

0.1058 

0.0634 

0.0318 

tilled_soil 

0.2982 

0.1248 

0.0678 

0.0193 

vamished_sand 

0.0176 

0.0050 

0.0106 

0.0020 

vamished_sandstone 

0.1255 

0.0163 

0.0650 

0.0334 

water 

0.1255 

0.0163 

0.0650 

0.0334 

wet_lakebed 

0.0607 

0.0180 

0.0259 

0.0032 

wet_seabed 

0.0389 

0.0279 

0.0623 

0.0727 

wet_soil 

0.0696 

0.0114 

0.0185 

0.0010 

white_sand 

0.0668 

0.0084 

0.0499 

0.0010 

Table  A-4:  Emissivity  mean  and  second  moment  values  for  different  types  cf  soils. 
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Vegetation 

3-5  pm  band 

8-12  pm  band 

E 

£ 

broadleaf 

0.0767 

0.0197 

0.0419 

0.0060 

broadleaf  (shaded) 

0.0804 

0.0180 

0.0726 

0.0059 

broadleaf-pine 

0.0405 

0.0032 

0.0395 

0.0031 

broadleaf-pine  (shaded) 

0.0380 

0 

0.0702 

0.0030 

broadleaf-scnib 

0.1880 

0.0729 

0.1072 

0.0410 

broadleaf-scrub  (shaded) 

0.1762 

0.0649 

0.1358 

0.0397 

dry_grass 

0.1447 

0.0534 

0.0756 

0.0240 

dry_grass  (shaded) 

0.1426 

0.0474 

0.1052 

0.0232 

grass-scrub 

0.1373 

0.0469 

0.0674 

0.0149 

grass-scrub  (shaded). 

0.1249 

0.0415 

0.0973 

0.0144 

grass-soil 

0.0385 

0.0026 

0.0371 

0.0024 

grass-soil  (shaded) 

0.0370 

0.0015 

0.0679 

0.0023 

lawn_grass 

0.0534 

0.0061 

0.0310 

0.0021 

lawn_grass  (shaded) 

0.0509 

0.0036 

0.0620 

0.0020 

pine 

0.0918 

0.0110 

0.0270 

0.0041 

pine  (shaded) 

0.0943 

0.0098 

0.0581 

0.0040 

pine-broadleaf 

0.0825 

0.0152 

0.0100 

0 

pine-broadleaf  (shaded) 

0.0747 

0.0077 

0.0417 

0 

pine-scrub 

0.1282 

0.0436 

0.0617 

0.0131 

pine-scrub  (shaded) 

0.1218 

0.0388 

0.0918 

0.0127 

scrub 

0.1025 

0.0316 

0.0421 

0.0017 

scrub  (shaded) 

0.0980 

0.0275 

0.0727 

0.0017 

scrub-grass-soil 

0.1012 

0.0344 

0.0438 

0.0083 

scrub-grass-soil  (shaded) 

0.1089 

0.0299 

0.0744 

0.0081 

tree_bark 

0.1012 

0.0344 

0.0438 

0.0083 

tree_bark  (shaded) 

0.1089 

0.0299 

0.0744 

0.0081 

tundra 

0.0967 

0.0198 

0.0527 

0.0040 

tundra  (shaded) 

0.0940 

0.0225 

0.0830 

0.0039 

Table  A-5:  Emissivity  mean  and  second  moment  values  for  different  types  of  vegetation. 


A.2  Error  Indicators  for  the  Atmospheric  Transmission  Coefficient 

The  El  values  for  spectral  atmospheric  transmission  coefficients  can  be  calculated  using 

£7^  =  200e^,  (A-2) 

where  is  the  second  moment  of  the  spectral  atmospheric  transmission  coefficient  data. 
The  second  moments  for  a  range  of  LOS  (line  of  sight)  paths  are  listed  in  Table  A-6. 
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Path  Description 

Range 

3-5  pm  band 

8  -  12  pm  band 

T 

X 

Horizontal  Path 

1  km 

0.5242 

0.3618 

0.6444 

0.1381 

2  km 

0.4179 

0.3466 

0.4458 

0.1399 

5  km 

X).27I2 

0.2881 

0.1650 

0.0943 

10  km 

0.1662 

0.2112 

0.0384 

0.0350 

20  km 

0.0786 

0.1198 

0.0029 

0.0041 

50  km 

0.0134 

0.0265 

0.0000 

0.0000 

Slant  Path  10°  Elevation 

1  km 

0.5290 

0.3618 

0.6572 

0.1374 

2km 

0.4283 

0.3483 

0.4784 

0.1411 

5  km 

0.2951 

0.2986 

0.2316 

0.1108 

10  km 

0.2107 

0.2457 

0.1076 

0.0716 

20  km 

0.1616 

0.2081 

0.0639 

0.0505 

50  km 

0.1298 

0.1803 

0.0520 

0.0436 

Slant  Path  20°  Elevation 

1  km 

0.5336 

0.3618 

0.6689 

0.1366 

2  km 

0.4378 

0.3496 

0.5062 

0.1416 

5  km 

0.3196 

0.3103 

0.2932 

0.1227 

10  km 

0.2635 

0.2820 

0.2157 

0.1069 

20  km 

0.2323 

0.2614 

0.1952 

0.0999 

50  km 

0.2147 

0.2480 

0.1823 

0.0963 

Slant  Path  30°  Elevation 

1  km 

0.5378 

0.3617 

0.6792 

0.1359 

2km 

0.4464 

0.3508 

0.5294 

0.1416 

5  km 

0.3447 

0.3216 

0.3554 

0.1319 

10  km 

0.3046 

0.3029 

0.3175 

0.1258 

20  km 

0.2819 

0.2891 

0.3026 

0.1216 

50  km 

0.2711 

0.2823 

0.2766 

0.1234 

Slant  Path  60°  Elevation 

1  km 

0.5471 

0.3614 

0.6995 

0.1339 

2km 

0.4667 

0.3544 

0.5785 

0.1417 

5  km 

0.3996 

0.3394 

0.4964 

0.1422 

10  km 

0.3742 

0.3292 

0.4793 

0.1400 

20  km 

0.3638 

0.3235 

0.4687 

0.1388 

50  km 

0.3587 

0.3215 

0.4382 

0.1463 

Slant  Path  90°  Elevation 

1  km 

0.5505 

0.3613 

0.7067 

0.1332 

2km 

0.4739 

0.3557 

0.5946 

0.1415 

5  km 

0.4157 

0.3434 

0.5343 

0.1429 

10  km 

0.3942 

0.3351 

0.5205 

0.1412 

20  km 

0.3861 

0.3310 

0.5057 

0.1416 

50  km 

0.3821 

0.3295 

0.4814 

0.1492 

Table  A-6:  Second  moment  and  mean  values  for  transmission  coefficients  of  some  LOS  paths. 
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A.3  Error  Indicators  for  Planck's  Blackbody  Spectral  Radiance 

The  El  value  for  Planck's  blackbody  spectral  radiance  is  defined  as 

~  (A-3) 

where  is  the  second  moment  of  the  normalised  Planck's  blackbody  spectral  radiance. 
The  blackbody  spectral  radiance  is  normalised  by  its  peak  value  in  the  wave  band  of 
interest.  The  second  moment  of  the  normalised  Planck's  blackbody  curve  as  a  function 
of  temperature  is  listed  in  Table  A-7. 


Temperature 

3-5  jim  band 

^bb 

8-12  pm  band 

^bb 

200 

0.2714 

0.1854 

225 

0.2806 

0.1409 

250 

0.2871 

0.0943 

275 

0.2910 

0.0490 

300 

0.2924 

0.0291 

325 

0.2912 

0.0531 

350 

0.2875 

0.0799 

375 

0.2814 

0.1006 

400 

0.2730 

0.1168 

425 

0.2626 

0.1297 

450 

0.2502 

0.1402 

475 

0.2360 

0.1488 

500 

0.2203 

0.1560 

525 

0.2032 

0.1620 

550 

0.1849 

0.1671 

575 

0.1656 

0.1715 

600 

0.1453 

0.1753 

625 

0.1237 

0.1786 

650 

0.1020 

0.1816 

675 

0.0811 

0.1842 

700 

0.0628 

0.1864 

725 

0.0493 

0.1885 

750 

0.0441 

0.1903 

775 

0.0483 

0.1920 

800 

0.0583 

0.1935 

825 

0.0706 

0.1949 

850 

0.0832 

0.1962 

875 

0.0952 

0.1973 

900 

0.1061 

0.1984 

925 

0.1159 

0.1994 

950 

0.1246 

0.2003 

975 

0.1320 

0.2011 

1000 

0.1387 

0.2019 

Table  A-7:  Second  moment  values  for  the  normalised  Planck's  blackbody  spectral  radiance. 
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A.4  Combining  Error  Indicator  Values 

The  error  indicator  associated  with  the  product  of  the  material  emissivity  and 
atmospheric  transmission  coefficient  is  given  by 

=  100  X  where  (A-4) 


Parameters  £  and  T  are  the  mean  emissivity  and  transmission  coefficient,  respectively, 
in  the  waveband  of  interest. 

The  total  error  indicator  including  the  effect  of  Planck's  blackbody  spectral  radiance  is 
calculated  using 


C'r 

X  Elf,. . 

100 


A.5  Equations  for  Calculating  the  Second  Moment 

The  second  moment  of  a  variable  is  given  by 

e  = 


e=  (A-8) 

»  n  =  1 

Equation  (A-7)  is  applicable  to  a  continuous  variable  Xj^  with  mean  value  x,  while  (A-8) 
is  applicable  to  a  discrete  variable  x^(n)  of  N  samples. 
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